Login
[x]
Log in using an account from:
Fedora Account System
Red Hat Associate
Red Hat Customer
Or login using a Red Hat Bugzilla account
Forgot Password
Login:
Hide Forgot
Create an Account
Red Hat Bugzilla – Attachment 944045 Details for
Bug 1124762
Unexpected error while setting up hosted-engine on EL7
[?]
New
Simple Search
Advanced Search
My Links
Browse
Requests
Reports
Current State
Search
Tabular reports
Graphical reports
Duplicates
Other Reports
User Changes
Plotly Reports
Bug Status
Bug Severity
Non-Defaults
|
Product Dashboard
Help
Page Help!
Bug Writing Guidelines
What's new
Browser Support Policy
5.0.4.rh83 Release notes
FAQ
Guides index
User guide
Web Services
Contact
Legal
This site requires JavaScript to be enabled to function correctly, please enable it.
vdsm.log
vdsm.log (text/plain), 16.46 MB, created by
Artyom
on 2014-10-05 14:37:20 UTC
(
hide
)
Description:
vdsm.log
Filename:
MIME Type:
Creator:
Artyom
Created:
2014-10-05 14:37:20 UTC
Size:
16.46 MB
patch
obsolete
>Thread-107427::WARNING::2014-10-02 20:01:11,445::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 20:01:11,448::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 20:01:11,449::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 20:01:21,450::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 20:01:21,450::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-02 20:01:23,036::__init__::232::IOProcess::(_processLogs) (91927) Queuing response >Thread-18::DEBUG::2014-10-02 20:01:23,037::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-02 20:01:23,037::__init__::232::IOProcess::(_processLogs) (91987) Got request for method 'glob' >Thread-107428::WARNING::2014-10-02 20:02:21,450::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 20:02:21,450::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 20:02:21,450::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-02 20:02:31,451::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-02 20:02:31,452::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 20:02:31,452::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107429::WARNING::2014-10-02 20:03:31,452::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 20:03:31,452::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 20:03:31,452::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 20:03:41,453::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 20:03:41,454::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107430::WARNING::2014-10-02 20:04:41,453::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 20:04:41,454::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 20:04:41,454::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 20:04:51,455::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 20:04:51,456::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107431::WARNING::2014-10-02 20:05:51,455::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 20:05:51,456::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 20:05:51,456::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 20:06:01,457::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 20:06:01,457::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107432::WARNING::2014-10-02 20:07:01,457::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 20:07:01,458::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 20:07:01,458::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 20:07:11,459::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 20:07:11,459::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107433::WARNING::2014-10-02 20:08:11,459::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 20:08:11,460::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 20:08:11,460::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-02 20:08:21,460::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-02 20:08:21,461::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 20:08:21,461::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107434::WARNING::2014-10-02 20:09:21,461::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 20:09:21,462::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 20:09:21,462::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 20:09:31,463::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 20:09:31,463::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107435::WARNING::2014-10-02 20:10:31,463::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 20:10:31,464::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 20:10:31,464::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 20:10:41,465::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 20:10:41,466::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-02 20:11:27,196::__init__::232::IOProcess::(_processLogs) (91928) Queuing response >Thread-18::DEBUG::2014-10-02 20:11:27,197::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-02 20:11:27,197::__init__::232::IOProcess::(_processLogs) (91988) Got request for method 'glob' >Thread-18::DEBUG::2014-10-02 20:11:27,197::__init__::232::IOProcess::(_processLogs) (91929) Queuing response >Thread-18::DEBUG::2014-10-02 20:11:27,197::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-02 20:11:27,197::__init__::232::IOProcess::(_processLogs) (91989) Got request for method 'glob' >Thread-107436::WARNING::2014-10-02 20:11:41,465::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 20:11:41,466::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 20:11:41,466::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 20:11:51,467::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 20:11:51,467::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107437::WARNING::2014-10-02 20:12:51,467::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 20:12:51,468::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 20:12:51,468::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 20:13:01,469::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 20:13:01,470::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107438::WARNING::2014-10-02 20:14:01,469::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 20:14:01,470::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 20:14:01,470::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-02 20:14:11,470::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-02 20:14:11,471::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 20:14:11,471::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107439::WARNING::2014-10-02 20:15:11,471::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 20:15:11,472::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 20:15:11,472::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 20:15:21,473::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 20:15:21,473::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107440::WARNING::2014-10-02 20:16:21,473::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 20:16:21,473::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 20:16:21,474::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 20:16:31,475::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 20:16:31,475::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107441::WARNING::2014-10-02 20:17:31,475::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 20:17:31,475::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 20:17:31,476::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 20:17:41,477::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 20:17:41,477::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107442::WARNING::2014-10-02 20:18:41,477::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 20:18:41,477::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 20:18:41,477::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 20:18:51,479::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 20:18:51,479::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107443::WARNING::2014-10-02 20:19:51,479::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 20:19:51,479::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 20:19:51,479::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-02 20:20:01,480::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-02 20:20:01,481::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 20:20:01,481::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107444::WARNING::2014-10-02 20:21:01,481::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 20:21:01,481::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 20:21:01,481::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 20:21:11,483::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 20:21:11,483::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-02 20:21:31,356::__init__::232::IOProcess::(_processLogs) (91930) Queuing response >Thread-18::DEBUG::2014-10-02 20:21:31,357::__init__::232::IOProcess::(_processLogs) (91931) Queuing response >Thread-18::DEBUG::2014-10-02 20:21:31,357::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-02 20:21:31,357::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-02 20:21:31,357::__init__::232::IOProcess::(_processLogs) (91990) Got request for method 'glob' >Thread-18::DEBUG::2014-10-02 20:21:31,357::__init__::232::IOProcess::(_processLogs) (91991) Got request for method 'glob' >Thread-107445::WARNING::2014-10-02 20:22:11,483::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 20:22:11,483::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 20:22:11,483::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 20:22:21,484::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 20:22:21,485::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107446::WARNING::2014-10-02 20:23:21,484::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 20:23:21,485::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 20:23:21,485::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 20:23:31,486::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 20:23:31,486::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107447::WARNING::2014-10-02 20:24:31,486::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 20:24:31,487::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 20:24:31,487::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 20:24:41,488::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 20:24:41,489::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107448::WARNING::2014-10-02 20:25:41,488::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 20:25:41,489::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 20:25:41,489::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-02 20:25:51,489::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-02 20:25:51,490::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 20:25:51,491::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107449::WARNING::2014-10-02 20:26:51,490::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 20:26:51,491::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 20:26:51,491::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 20:27:01,492::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 20:27:01,492::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107450::WARNING::2014-10-02 20:28:01,492::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 20:28:01,493::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 20:28:01,493::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 20:28:11,494::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 20:28:11,494::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107451::WARNING::2014-10-02 20:29:11,494::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 20:29:11,495::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 20:29:11,495::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 20:29:21,496::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 20:29:21,496::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107452::WARNING::2014-10-02 20:30:21,496::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 20:30:21,497::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 20:30:21,497::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 20:30:31,498::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 20:30:31,498::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107453::WARNING::2014-10-02 20:31:31,498::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 20:31:31,499::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 20:31:31,499::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 20:31:37,564::__init__::232::IOProcess::(_processLogs) (91932) Queuing response >Thread-18::DEBUG::2014-10-02 20:31:37,565::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-02 20:31:37,565::__init__::232::IOProcess::(_processLogs) (91992) Got request for method 'glob' >Thread-18::DEBUG::2014-10-02 20:31:37,565::__init__::232::IOProcess::(_processLogs) (91933) Queuing response >Thread-18::DEBUG::2014-10-02 20:31:37,565::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-02 20:31:37,565::__init__::232::IOProcess::(_processLogs) (91993) Got request for method 'glob' >Thread-48::DEBUG::2014-10-02 20:31:41,499::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-02 20:31:41,500::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 20:31:41,500::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107454::WARNING::2014-10-02 20:32:41,500::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 20:32:41,501::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 20:32:41,501::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 20:32:51,502::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 20:32:51,503::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107455::WARNING::2014-10-02 20:33:51,502::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 20:33:51,503::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 20:33:51,503::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 20:34:01,504::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 20:34:01,505::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107456::WARNING::2014-10-02 20:35:01,504::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 20:35:01,505::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 20:35:01,505::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 20:35:11,506::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 20:35:11,506::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107457::WARNING::2014-10-02 20:36:11,506::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 20:36:11,507::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 20:36:11,507::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 20:36:21,508::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 20:36:21,508::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107458::WARNING::2014-10-02 20:37:21,508::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 20:37:21,509::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 20:37:21,509::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-02 20:37:31,509::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-02 20:37:31,510::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 20:37:31,510::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107459::WARNING::2014-10-02 20:38:31,510::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 20:38:31,511::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 20:38:31,511::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 20:38:41,512::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 20:38:41,513::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107460::WARNING::2014-10-02 20:39:41,512::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 20:39:41,513::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 20:39:41,513::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 20:39:51,514::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 20:39:51,515::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107461::WARNING::2014-10-02 20:40:51,514::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 20:40:51,515::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 20:40:51,515::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 20:41:01,516::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 20:41:01,517::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-02 20:41:41,724::__init__::232::IOProcess::(_processLogs) (91934) Queuing response >Thread-18::DEBUG::2014-10-02 20:41:41,725::__init__::232::IOProcess::(_processLogs) (91935) Queuing response >Thread-18::DEBUG::2014-10-02 20:41:41,725::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-02 20:41:41,725::__init__::232::IOProcess::(_processLogs) (91994) Got request for method 'glob' >Thread-18::DEBUG::2014-10-02 20:41:41,725::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-02 20:41:41,725::__init__::232::IOProcess::(_processLogs) (91995) Got request for method 'glob' >Thread-107462::WARNING::2014-10-02 20:42:01,516::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 20:42:01,517::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 20:42:01,517::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 20:42:11,518::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 20:42:11,519::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107463::WARNING::2014-10-02 20:43:11,518::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 20:43:11,519::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 20:43:11,519::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-02 20:43:21,520::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-02 20:43:21,521::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 20:43:21,521::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107464::WARNING::2014-10-02 20:44:21,520::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 20:44:21,521::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 20:44:21,521::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 20:44:31,522::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 20:44:31,523::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107465::WARNING::2014-10-02 20:45:31,522::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 20:45:31,523::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 20:45:31,523::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 20:45:41,524::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 20:45:41,524::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107466::WARNING::2014-10-02 20:46:41,524::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 20:46:41,525::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 20:46:41,525::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 20:46:51,526::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 20:46:51,526::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107467::WARNING::2014-10-02 20:47:51,526::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 20:47:51,526::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 20:47:51,526::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 20:48:01,528::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 20:48:01,528::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107468::WARNING::2014-10-02 20:49:01,528::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 20:49:01,528::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 20:49:01,528::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-02 20:49:11,529::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-02 20:49:11,530::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 20:49:11,530::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107469::WARNING::2014-10-02 20:50:11,530::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 20:50:11,530::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 20:50:11,530::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 20:50:21,531::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 20:50:21,532::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107470::WARNING::2014-10-02 20:51:21,531::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 20:51:21,532::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 20:51:21,532::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 20:51:31,533::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 20:51:31,533::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-02 20:51:47,933::__init__::232::IOProcess::(_processLogs) (91936) Queuing response >Thread-18::DEBUG::2014-10-02 20:51:47,933::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-02 20:51:47,933::__init__::232::IOProcess::(_processLogs) (91996) Got request for method 'glob' >Thread-107471::WARNING::2014-10-02 20:52:31,533::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 20:52:31,534::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 20:52:31,534::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 20:52:41,535::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 20:52:41,535::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107472::WARNING::2014-10-02 20:53:41,535::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 20:53:41,536::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 20:53:41,536::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 20:53:51,537::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 20:53:51,537::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107473::WARNING::2014-10-02 20:54:51,537::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 20:54:51,538::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 20:54:51,538::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-02 20:55:01,538::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-02 20:55:01,539::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 20:55:01,539::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107474::WARNING::2014-10-02 20:56:01,539::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 20:56:01,540::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 20:56:01,540::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 20:56:11,541::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 20:56:11,541::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107475::WARNING::2014-10-02 20:57:11,541::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 20:57:11,541::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 20:57:11,542::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 20:57:21,543::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 20:57:21,543::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107476::WARNING::2014-10-02 20:58:21,543::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 20:58:21,543::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 20:58:21,543::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 20:58:31,545::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 20:58:31,545::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107477::WARNING::2014-10-02 20:59:31,544::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 20:59:31,545::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 20:59:31,545::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 20:59:41,546::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 20:59:41,546::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107478::WARNING::2014-10-02 21:00:41,546::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 21:00:41,547::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 21:00:41,547::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-02 21:00:51,548::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-02 21:00:51,548::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 21:00:51,549::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107479::WARNING::2014-10-02 21:01:51,548::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 21:01:51,549::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 21:01:51,549::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 21:01:54,140::__init__::232::IOProcess::(_processLogs) (91937) Queuing response >Thread-18::DEBUG::2014-10-02 21:01:54,141::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-02 21:01:54,141::__init__::232::IOProcess::(_processLogs) (91997) Got request for method 'glob' >Thread-18::DEBUG::2014-10-02 21:02:01,550::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 21:02:01,551::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107480::WARNING::2014-10-02 21:03:01,550::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 21:03:01,551::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 21:03:01,551::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 21:03:11,552::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 21:03:11,552::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107481::WARNING::2014-10-02 21:04:11,552::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 21:04:11,553::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 21:04:11,553::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 21:04:21,554::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 21:04:21,554::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107482::WARNING::2014-10-02 21:05:21,554::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 21:05:21,555::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 21:05:21,555::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 21:05:31,556::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 21:05:31,556::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107483::WARNING::2014-10-02 21:06:31,556::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 21:06:31,557::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 21:06:31,557::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-02 21:06:41,557::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-02 21:06:41,558::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 21:06:41,558::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107484::WARNING::2014-10-02 21:07:41,558::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 21:07:41,558::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 21:07:41,559::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 21:07:51,560::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 21:07:51,560::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107485::WARNING::2014-10-02 21:08:51,560::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 21:08:51,560::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 21:08:51,560::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 21:09:01,562::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 21:09:01,562::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107486::WARNING::2014-10-02 21:10:01,562::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 21:10:01,562::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 21:10:01,562::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 21:10:11,563::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 21:10:11,564::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107487::WARNING::2014-10-02 21:11:11,563::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 21:11:11,564::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 21:11:11,564::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 21:11:21,565::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 21:11:21,565::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-02 21:11:58,301::__init__::232::IOProcess::(_processLogs) (91938) Queuing response >Thread-18::DEBUG::2014-10-02 21:11:58,301::__init__::232::IOProcess::(_processLogs) (91939) Queuing response >Thread-18::DEBUG::2014-10-02 21:11:58,301::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-02 21:11:58,301::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-02 21:11:58,301::__init__::232::IOProcess::(_processLogs) (91998) Got request for method 'glob' >Thread-18::DEBUG::2014-10-02 21:11:58,301::__init__::232::IOProcess::(_processLogs) (91999) Got request for method 'glob' >Thread-107488::WARNING::2014-10-02 21:12:21,565::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 21:12:21,567::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 21:12:21,567::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-02 21:12:31,568::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-02 21:12:31,569::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 21:12:31,569::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107489::WARNING::2014-10-02 21:13:31,569::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 21:13:31,570::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 21:13:31,570::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 21:13:41,571::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 21:13:41,572::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107490::WARNING::2014-10-02 21:14:41,571::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 21:14:41,572::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 21:14:41,572::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 21:14:51,574::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 21:14:51,574::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107491::WARNING::2014-10-02 21:15:51,573::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 21:15:51,574::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 21:15:51,574::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 21:16:01,575::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 21:16:01,575::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107492::WARNING::2014-10-02 21:17:01,575::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 21:17:01,576::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 21:17:01,576::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 21:17:11,577::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 21:17:11,577::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107493::WARNING::2014-10-02 21:18:11,577::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 21:18:11,577::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 21:18:11,578::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-02 21:18:21,578::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-02 21:18:21,579::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 21:18:21,579::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107494::WARNING::2014-10-02 21:19:21,579::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 21:19:21,579::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 21:19:21,580::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 21:19:31,581::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 21:19:31,581::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107495::WARNING::2014-10-02 21:20:31,581::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 21:20:31,581::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 21:20:31,581::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 21:20:41,583::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 21:20:41,583::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107496::WARNING::2014-10-02 21:21:41,583::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 21:21:41,584::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 21:21:41,584::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 21:21:51,586::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 21:21:51,586::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-02 21:22:02,460::__init__::232::IOProcess::(_processLogs) (91940) Queuing response >Thread-18::DEBUG::2014-10-02 21:22:02,461::__init__::232::IOProcess::(_processLogs) (91941) Queuing response >Thread-18::DEBUG::2014-10-02 21:22:02,461::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-02 21:22:02,461::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-02 21:22:02,461::__init__::232::IOProcess::(_processLogs) (92000) Got request for method 'glob' >Thread-18::DEBUG::2014-10-02 21:22:02,461::__init__::232::IOProcess::(_processLogs) (92001) Got request for method 'glob' >Thread-107497::WARNING::2014-10-02 21:22:51,586::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 21:22:51,586::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 21:22:51,586::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 21:23:01,588::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 21:23:01,588::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107498::WARNING::2014-10-02 21:24:01,588::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 21:24:01,588::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 21:24:01,588::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-02 21:24:11,589::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-02 21:24:11,590::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 21:24:11,590::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107499::WARNING::2014-10-02 21:25:11,590::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 21:25:11,590::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 21:25:11,590::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 21:25:21,591::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 21:25:21,592::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107500::WARNING::2014-10-02 21:26:21,591::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 21:26:21,592::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 21:26:21,592::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 21:26:31,593::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 21:26:31,593::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107501::WARNING::2014-10-02 21:27:31,593::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 21:27:31,594::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 21:27:31,594::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 21:27:41,595::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 21:27:41,595::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107502::WARNING::2014-10-02 21:28:41,595::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 21:28:41,595::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 21:28:41,596::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 21:28:51,597::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 21:28:51,597::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107503::WARNING::2014-10-02 21:29:51,597::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 21:29:51,597::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 21:29:51,598::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-02 21:30:01,598::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-02 21:30:01,599::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 21:30:01,599::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107504::WARNING::2014-10-02 21:31:01,599::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 21:31:01,599::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 21:31:01,600::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 21:31:11,601::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 21:31:11,602::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-02 21:32:06,620::__init__::232::IOProcess::(_processLogs) (91942) Queuing response >Thread-18::DEBUG::2014-10-02 21:32:06,621::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-02 21:32:06,621::__init__::232::IOProcess::(_processLogs) (92002) Got request for method 'glob' >Thread-18::DEBUG::2014-10-02 21:32:06,621::__init__::232::IOProcess::(_processLogs) (91943) Queuing response >Thread-18::DEBUG::2014-10-02 21:32:06,621::__init__::232::IOProcess::(_processLogs) (91944) Queuing response >Thread-18::DEBUG::2014-10-02 21:32:06,621::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-02 21:32:06,621::__init__::232::IOProcess::(_processLogs) (92003) Got request for method 'glob' >Thread-18::DEBUG::2014-10-02 21:32:06,621::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-02 21:32:06,622::__init__::232::IOProcess::(_processLogs) (92004) Got request for method 'glob' >Thread-107505::WARNING::2014-10-02 21:32:11,601::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 21:32:11,602::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 21:32:11,602::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 21:32:21,603::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 21:32:21,603::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107506::WARNING::2014-10-02 21:33:21,603::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 21:33:21,604::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 21:33:21,604::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 21:33:31,605::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 21:33:31,605::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107507::WARNING::2014-10-02 21:34:31,605::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 21:34:31,606::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 21:34:31,606::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 21:34:41,607::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 21:34:41,607::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107508::WARNING::2014-10-02 21:35:41,607::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 21:35:41,608::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 21:35:41,608::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-02 21:35:51,608::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-02 21:35:51,609::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 21:35:51,609::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107509::WARNING::2014-10-02 21:36:51,609::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 21:36:51,610::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 21:36:51,610::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 21:37:01,611::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 21:37:01,611::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107510::WARNING::2014-10-02 21:38:01,611::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 21:38:01,612::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 21:38:01,612::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 21:38:11,613::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 21:38:11,613::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107511::WARNING::2014-10-02 21:39:11,613::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 21:39:11,613::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 21:39:11,614::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 21:39:21,615::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 21:39:21,615::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107512::WARNING::2014-10-02 21:40:21,615::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 21:40:21,615::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 21:40:21,616::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 21:40:31,617::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 21:40:31,617::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107513::WARNING::2014-10-02 21:41:31,617::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 21:41:31,617::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 21:41:31,618::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-02 21:41:41,618::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-02 21:41:41,619::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 21:41:41,619::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-02 21:42:10,780::__init__::232::IOProcess::(_processLogs) (91945) Queuing response >Thread-18::DEBUG::2014-10-02 21:42:10,781::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-02 21:42:10,781::__init__::232::IOProcess::(_processLogs) (92005) Got request for method 'glob' >Thread-107514::WARNING::2014-10-02 21:42:41,619::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 21:42:41,620::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 21:42:41,620::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 21:42:51,621::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 21:42:51,621::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107515::WARNING::2014-10-02 21:43:51,621::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 21:43:51,622::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 21:43:51,622::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 21:44:01,623::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 21:44:01,623::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107516::WARNING::2014-10-02 21:45:01,623::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 21:45:01,623::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 21:45:01,624::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 21:45:11,625::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 21:45:11,625::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107517::WARNING::2014-10-02 21:46:11,625::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 21:46:11,625::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 21:46:11,625::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 21:46:21,626::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 21:46:21,627::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107518::WARNING::2014-10-02 21:47:21,626::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 21:47:21,627::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 21:47:21,627::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-02 21:47:31,627::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-02 21:47:31,628::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 21:47:31,629::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107519::WARNING::2014-10-02 21:48:31,628::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 21:48:31,629::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 21:48:31,629::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 21:48:41,630::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 21:48:41,630::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107520::WARNING::2014-10-02 21:49:41,630::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 21:49:41,630::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 21:49:41,631::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 21:49:51,632::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 21:49:51,632::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107521::WARNING::2014-10-02 21:50:51,632::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 21:50:51,632::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 21:50:51,632::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 21:51:01,634::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 21:51:01,634::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107522::WARNING::2014-10-02 21:52:01,634::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 21:52:01,634::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 21:52:01,634::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 21:52:11,635::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 21:52:11,636::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-02 21:52:14,940::__init__::232::IOProcess::(_processLogs) (91946) Queuing response >Thread-18::DEBUG::2014-10-02 21:52:14,941::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-02 21:52:14,941::__init__::232::IOProcess::(_processLogs) (92006) Got request for method 'glob' >Thread-107523::WARNING::2014-10-02 21:53:11,635::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 21:53:11,636::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 21:53:11,636::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-02 21:53:21,636::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-02 21:53:21,637::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 21:53:21,637::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107524::WARNING::2014-10-02 21:54:21,637::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 21:54:21,638::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 21:54:21,638::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 21:54:31,639::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 21:54:31,639::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107525::WARNING::2014-10-02 21:55:31,639::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 21:55:31,639::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 21:55:31,640::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 21:55:41,641::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 21:55:41,641::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107526::WARNING::2014-10-02 21:56:41,641::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 21:56:41,641::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 21:56:41,641::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 21:56:51,643::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 21:56:51,643::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107527::WARNING::2014-10-02 21:57:51,642::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 21:57:51,643::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 21:57:51,643::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 21:58:01,644::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 21:58:01,644::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107528::WARNING::2014-10-02 21:59:01,644::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 21:59:01,645::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 21:59:01,645::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-02 21:59:11,645::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-02 21:59:11,646::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 21:59:11,646::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107529::WARNING::2014-10-02 22:00:11,646::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 22:00:11,647::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 22:00:11,647::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 22:00:21,648::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 22:00:21,648::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107530::WARNING::2014-10-02 22:01:21,648::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 22:01:21,648::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 22:01:21,648::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 22:01:31,650::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 22:01:31,650::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-02 22:02:19,100::__init__::232::IOProcess::(_processLogs) (91947) Queuing response >Thread-18::DEBUG::2014-10-02 22:02:19,101::__init__::232::IOProcess::(_processLogs) (91948) Queuing response >Thread-18::DEBUG::2014-10-02 22:02:19,101::__init__::232::IOProcess::(_processLogs) (91949) Queuing response >Thread-18::DEBUG::2014-10-02 22:02:19,101::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-02 22:02:19,101::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-02 22:02:19,101::__init__::232::IOProcess::(_processLogs) (92008) Got request for method 'glob' >Thread-18::DEBUG::2014-10-02 22:02:19,101::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-02 22:02:19,101::__init__::232::IOProcess::(_processLogs) (92007) Got request for method 'glob' >Thread-18::DEBUG::2014-10-02 22:02:19,101::__init__::232::IOProcess::(_processLogs) (92009) Got request for method 'glob' >Thread-107531::WARNING::2014-10-02 22:02:31,650::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 22:02:31,650::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 22:02:31,650::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 22:02:41,651::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 22:02:41,651::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107532::WARNING::2014-10-02 22:03:41,651::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 22:03:41,652::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 22:03:41,652::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 22:03:51,653::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 22:03:51,653::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107533::WARNING::2014-10-02 22:04:51,653::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 22:04:51,653::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 22:04:51,654::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-02 22:05:01,654::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-02 22:05:01,655::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 22:05:01,655::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107534::WARNING::2014-10-02 22:06:01,655::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 22:06:01,655::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 22:06:01,656::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 22:06:11,657::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 22:06:11,657::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107535::WARNING::2014-10-02 22:07:11,657::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 22:07:11,657::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 22:07:11,657::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 22:07:21,658::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 22:07:21,659::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107536::WARNING::2014-10-02 22:08:21,658::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 22:08:21,659::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 22:08:21,659::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 22:08:31,660::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 22:08:31,661::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107537::WARNING::2014-10-02 22:09:31,660::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 22:09:31,661::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 22:09:31,661::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 22:09:41,662::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 22:09:41,663::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107538::WARNING::2014-10-02 22:10:41,662::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 22:10:41,663::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 22:10:41,663::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-02 22:10:51,664::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-02 22:10:51,665::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 22:10:51,665::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107539::WARNING::2014-10-02 22:11:51,665::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 22:11:51,665::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 22:11:51,665::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 22:12:01,667::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 22:12:01,667::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-02 22:12:23,260::__init__::232::IOProcess::(_processLogs) (91950) Queuing response >Thread-18::DEBUG::2014-10-02 22:12:23,261::__init__::232::IOProcess::(_processLogs) (91952) Queuing response >Thread-18::DEBUG::2014-10-02 22:12:23,261::__init__::232::IOProcess::(_processLogs) (91951) Queuing response >Thread-18::DEBUG::2014-10-02 22:12:23,261::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-02 22:12:23,261::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-02 22:12:23,261::__init__::232::IOProcess::(_processLogs) (92011) Got request for method 'glob' >Thread-18::DEBUG::2014-10-02 22:12:23,261::__init__::232::IOProcess::(_processLogs) (92010) Got request for method 'glob' >Thread-18::DEBUG::2014-10-02 22:12:23,261::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-02 22:12:23,261::__init__::232::IOProcess::(_processLogs) (92012) Got request for method 'glob' >Thread-107540::WARNING::2014-10-02 22:13:01,667::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 22:13:01,667::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 22:13:01,667::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 22:13:11,669::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 22:13:11,669::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107541::WARNING::2014-10-02 22:14:11,669::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 22:14:11,670::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 22:14:11,670::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 22:14:21,671::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 22:14:21,671::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107542::WARNING::2014-10-02 22:15:21,671::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 22:15:21,671::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 22:15:21,672::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 22:15:31,673::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 22:15:31,673::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107543::WARNING::2014-10-02 22:16:31,673::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 22:16:31,673::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 22:16:31,673::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-02 22:16:41,674::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-02 22:16:41,675::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 22:16:41,675::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107544::WARNING::2014-10-02 22:17:41,675::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 22:17:41,675::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 22:17:41,675::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 22:17:51,677::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 22:17:51,677::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107545::WARNING::2014-10-02 22:18:51,677::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 22:18:51,677::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 22:18:51,677::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 22:19:01,678::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 22:19:01,679::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107546::WARNING::2014-10-02 22:20:01,678::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 22:20:01,679::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 22:20:01,679::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 22:20:11,680::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 22:20:11,681::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107547::WARNING::2014-10-02 22:21:11,680::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 22:21:11,681::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 22:21:11,681::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 22:21:21,682::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 22:21:21,682::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107548::WARNING::2014-10-02 22:22:21,682::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 22:22:21,683::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 22:22:21,683::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 22:22:27,420::__init__::232::IOProcess::(_processLogs) (91953) Queuing response >Thread-18::DEBUG::2014-10-02 22:22:27,421::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-02 22:22:27,421::__init__::232::IOProcess::(_processLogs) (92013) Got request for method 'glob' >Thread-18::DEBUG::2014-10-02 22:22:27,421::__init__::232::IOProcess::(_processLogs) (91954) Queuing response >Thread-18::DEBUG::2014-10-02 22:22:27,421::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-02 22:22:27,421::__init__::232::IOProcess::(_processLogs) (92014) Got request for method 'glob' >Thread-48::DEBUG::2014-10-02 22:22:31,683::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-02 22:22:31,684::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 22:22:31,684::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107549::WARNING::2014-10-02 22:23:31,684::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 22:23:31,685::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 22:23:31,685::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 22:23:41,686::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 22:23:41,686::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107550::WARNING::2014-10-02 22:24:41,686::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 22:24:41,686::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 22:24:41,687::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 22:24:51,688::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 22:24:51,688::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107551::WARNING::2014-10-02 22:25:51,688::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 22:25:51,688::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 22:25:51,688::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 22:26:01,689::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 22:26:01,690::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107552::WARNING::2014-10-02 22:27:01,689::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 22:27:01,690::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 22:27:01,690::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 22:27:11,691::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 22:27:11,691::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107553::WARNING::2014-10-02 22:28:11,691::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 22:28:11,692::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 22:28:11,692::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-02 22:28:21,692::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-02 22:28:21,693::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 22:28:21,693::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107554::WARNING::2014-10-02 22:29:21,693::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 22:29:21,694::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 22:29:21,694::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 22:29:31,695::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 22:29:31,695::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107555::WARNING::2014-10-02 22:30:31,695::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 22:30:31,696::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 22:30:31,696::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 22:30:41,697::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 22:30:41,697::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107556::WARNING::2014-10-02 22:31:41,697::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 22:31:41,697::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 22:31:41,698::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 22:31:51,699::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 22:31:51,699::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-02 22:32:33,628::__init__::232::IOProcess::(_processLogs) (91955) Queuing response >Thread-18::DEBUG::2014-10-02 22:32:33,629::__init__::232::IOProcess::(_processLogs) (91956) Queuing response >Thread-18::DEBUG::2014-10-02 22:32:33,629::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-02 22:32:33,629::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-02 22:32:33,629::__init__::232::IOProcess::(_processLogs) (92015) Got request for method 'glob' >Thread-18::DEBUG::2014-10-02 22:32:33,629::__init__::232::IOProcess::(_processLogs) (92016) Got request for method 'glob' >Thread-107557::WARNING::2014-10-02 22:32:51,699::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 22:32:51,699::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 22:32:51,700::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 22:33:01,701::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 22:33:01,701::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107558::WARNING::2014-10-02 22:34:01,701::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 22:34:01,701::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 22:34:01,701::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-02 22:34:11,702::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-02 22:34:11,703::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 22:34:11,703::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107559::WARNING::2014-10-02 22:35:11,703::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 22:35:11,703::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 22:35:11,703::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 22:35:21,705::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 22:35:21,705::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107560::WARNING::2014-10-02 22:36:21,704::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 22:36:21,705::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 22:36:21,705::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 22:36:31,706::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 22:36:31,707::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107561::WARNING::2014-10-02 22:37:31,706::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 22:37:31,707::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 22:37:31,707::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 22:37:41,708::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 22:37:41,708::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107562::WARNING::2014-10-02 22:38:41,708::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 22:38:41,709::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 22:38:41,709::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 22:38:51,710::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 22:38:51,710::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107563::WARNING::2014-10-02 22:39:51,710::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 22:39:51,711::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 22:39:51,711::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-02 22:40:01,711::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-02 22:40:01,712::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 22:40:01,713::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107564::WARNING::2014-10-02 22:41:01,712::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 22:41:01,713::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 22:41:01,713::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 22:41:11,714::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 22:41:11,715::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107565::WARNING::2014-10-02 22:42:11,714::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 22:42:11,715::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 22:42:11,715::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 22:42:21,716::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 22:42:21,717::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-02 22:42:39,836::__init__::232::IOProcess::(_processLogs) (91957) Queuing response >Thread-18::DEBUG::2014-10-02 22:42:39,837::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-02 22:42:39,837::__init__::232::IOProcess::(_processLogs) (92017) Got request for method 'glob' >Thread-18::DEBUG::2014-10-02 22:42:39,837::__init__::232::IOProcess::(_processLogs) (91958) Queuing response >Thread-18::DEBUG::2014-10-02 22:42:39,837::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-02 22:42:39,837::__init__::232::IOProcess::(_processLogs) (92018) Got request for method 'glob' >Thread-107566::WARNING::2014-10-02 22:43:21,716::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 22:43:21,717::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 22:43:21,717::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 22:43:31,718::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 22:43:31,719::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107567::WARNING::2014-10-02 22:44:31,718::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 22:44:31,719::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 22:44:31,719::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 22:44:41,720::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 22:44:41,720::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107568::WARNING::2014-10-02 22:45:41,720::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 22:45:41,720::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 22:45:41,721::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-02 22:45:51,721::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-02 22:45:51,722::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 22:45:51,722::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107569::WARNING::2014-10-02 22:46:51,722::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 22:46:51,722::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 22:46:51,723::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 22:47:01,724::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 22:47:01,724::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107570::WARNING::2014-10-02 22:48:01,724::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 22:48:01,724::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 22:48:01,724::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 22:48:11,725::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 22:48:11,726::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107571::WARNING::2014-10-02 22:49:11,725::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 22:49:11,726::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 22:49:11,726::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 22:49:21,727::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 22:49:21,727::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107572::WARNING::2014-10-02 22:50:21,727::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 22:50:21,728::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 22:50:21,728::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 22:50:31,729::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 22:50:31,729::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107573::WARNING::2014-10-02 22:51:31,729::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 22:51:31,729::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 22:51:31,730::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-02 22:51:41,730::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-02 22:51:41,731::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 22:51:41,731::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107574::WARNING::2014-10-02 22:52:41,731::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 22:52:41,731::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 22:52:41,731::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 22:52:46,044::__init__::232::IOProcess::(_processLogs) (91959) Queuing response >Thread-18::DEBUG::2014-10-02 22:52:46,045::__init__::232::IOProcess::(_processLogs) (91960) Queuing response >Thread-18::DEBUG::2014-10-02 22:52:46,045::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-02 22:52:46,045::__init__::232::IOProcess::(_processLogs) (92019) Got request for method 'glob' >Thread-18::DEBUG::2014-10-02 22:52:46,045::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-02 22:52:46,045::__init__::232::IOProcess::(_processLogs) (92020) Got request for method 'glob' >Thread-18::DEBUG::2014-10-02 22:52:46,045::__init__::232::IOProcess::(_processLogs) (91961) Queuing response >Thread-18::DEBUG::2014-10-02 22:52:46,045::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-02 22:52:46,045::__init__::232::IOProcess::(_processLogs) (92021) Got request for method 'glob' >Thread-18::DEBUG::2014-10-02 22:52:51,733::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 22:52:51,733::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107575::WARNING::2014-10-02 22:53:51,732::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 22:53:51,733::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 22:53:51,733::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 22:54:01,734::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 22:54:01,734::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107576::WARNING::2014-10-02 22:55:01,734::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 22:55:01,735::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 22:55:01,735::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 22:55:11,736::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 22:55:11,736::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107577::WARNING::2014-10-02 22:56:11,736::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 22:56:11,736::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 22:56:11,737::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 22:56:21,738::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 22:56:21,738::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107578::WARNING::2014-10-02 22:57:21,738::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 22:57:21,738::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 22:57:21,738::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-02 22:57:31,739::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-02 22:57:31,740::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 22:57:31,740::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107579::WARNING::2014-10-02 22:58:31,740::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 22:58:31,740::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 22:58:31,740::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 22:58:41,741::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 22:58:41,742::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107580::WARNING::2014-10-02 22:59:41,741::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 22:59:41,742::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 22:59:41,742::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 22:59:51,743::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 22:59:51,743::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107581::WARNING::2014-10-02 23:00:51,743::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 23:00:51,743::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 23:00:51,744::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 23:01:01,745::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 23:01:01,745::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107582::WARNING::2014-10-02 23:02:01,745::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 23:02:01,745::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 23:02:01,745::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 23:02:11,746::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 23:02:11,747::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-02 23:02:50,205::__init__::232::IOProcess::(_processLogs) (91963) Queuing response >Thread-18::DEBUG::2014-10-02 23:02:50,205::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-02 23:02:50,206::__init__::232::IOProcess::(_processLogs) (92022) Got request for method 'glob' >Thread-18::DEBUG::2014-10-02 23:02:50,206::__init__::232::IOProcess::(_processLogs) (91962) Queuing response >Thread-18::DEBUG::2014-10-02 23:02:50,207::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-02 23:02:50,207::__init__::232::IOProcess::(_processLogs) (92023) Got request for method 'glob' >Thread-107583::WARNING::2014-10-02 23:03:11,746::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 23:03:11,747::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 23:03:11,747::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-02 23:03:21,747::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-02 23:03:21,748::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 23:03:21,749::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107584::WARNING::2014-10-02 23:04:21,748::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 23:04:21,749::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 23:04:21,749::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 23:04:31,750::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 23:04:31,750::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107585::WARNING::2014-10-02 23:05:31,750::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 23:05:31,751::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 23:05:31,751::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 23:05:41,752::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 23:05:41,752::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107586::WARNING::2014-10-02 23:06:41,752::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 23:06:41,753::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 23:06:41,753::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 23:06:51,754::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 23:06:51,754::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107587::WARNING::2014-10-02 23:07:51,754::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 23:07:51,754::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 23:07:51,755::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 23:08:01,756::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 23:08:01,756::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107588::WARNING::2014-10-02 23:09:01,756::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 23:09:01,756::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 23:09:01,756::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-02 23:09:11,757::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-02 23:09:11,758::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 23:09:11,758::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107589::WARNING::2014-10-02 23:10:11,758::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 23:10:11,758::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 23:10:11,759::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 23:10:21,760::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 23:10:21,760::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107590::WARNING::2014-10-02 23:11:21,760::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 23:11:21,760::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 23:11:21,760::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 23:11:31,762::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 23:11:31,762::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107591::WARNING::2014-10-02 23:12:31,762::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 23:12:31,762::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 23:12:31,762::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 23:12:41,764::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 23:12:41,764::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107592::WARNING::2014-10-02 23:13:41,764::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 23:13:41,764::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 23:13:41,764::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 23:13:51,766::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 23:13:51,766::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107593::WARNING::2014-10-02 23:14:51,766::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 23:14:51,766::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 23:14:51,766::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-02 23:15:01,767::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-02 23:15:01,768::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 23:15:01,768::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107594::WARNING::2014-10-02 23:16:01,768::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 23:16:01,768::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 23:16:01,768::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 23:16:11,770::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 23:16:11,770::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107595::WARNING::2014-10-02 23:17:11,769::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 23:17:11,770::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 23:17:11,770::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 23:17:21,771::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 23:17:21,771::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107596::WARNING::2014-10-02 23:18:21,771::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 23:18:21,772::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 23:18:21,772::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 23:18:31,773::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 23:18:31,774::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107597::WARNING::2014-10-02 23:19:31,773::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 23:19:31,774::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 23:19:31,774::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 23:19:41,775::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 23:19:41,775::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107598::WARNING::2014-10-02 23:20:41,775::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 23:20:41,776::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 23:20:41,776::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-02 23:20:51,776::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-02 23:20:51,777::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 23:20:51,777::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107599::WARNING::2014-10-02 23:21:51,777::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 23:21:51,778::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 23:21:51,778::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 23:22:01,779::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 23:22:01,779::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107600::WARNING::2014-10-02 23:23:01,779::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 23:23:01,779::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 23:23:01,780::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 23:23:11,781::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 23:23:11,781::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107601::WARNING::2014-10-02 23:24:11,781::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 23:24:11,781::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 23:24:11,782::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 23:24:21,783::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 23:24:21,783::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107602::WARNING::2014-10-02 23:25:21,783::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 23:25:21,783::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 23:25:21,783::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 23:25:31,785::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 23:25:31,785::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107603::WARNING::2014-10-02 23:26:31,785::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 23:26:31,785::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 23:26:31,785::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-02 23:26:41,786::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-02 23:26:41,787::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 23:26:41,787::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107604::WARNING::2014-10-02 23:27:41,787::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 23:27:41,787::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 23:27:41,787::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 23:27:51,788::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 23:27:51,789::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107605::WARNING::2014-10-02 23:28:51,788::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 23:28:51,789::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 23:28:51,789::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 23:29:01,790::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 23:29:01,790::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107606::WARNING::2014-10-02 23:30:01,790::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 23:30:01,791::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 23:30:01,791::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 23:30:11,792::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 23:30:11,792::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107607::WARNING::2014-10-02 23:31:11,792::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 23:31:11,793::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 23:31:11,793::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 23:31:21,794::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 23:31:21,794::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107608::WARNING::2014-10-02 23:32:21,794::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 23:32:21,795::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 23:32:21,795::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-02 23:32:31,795::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-02 23:32:31,796::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 23:32:31,796::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107609::WARNING::2014-10-02 23:33:31,796::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 23:33:31,797::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 23:33:31,797::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 23:33:41,798::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 23:33:41,798::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107610::WARNING::2014-10-02 23:34:41,798::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 23:34:41,798::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 23:34:41,799::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 23:34:51,800::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 23:34:51,800::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107611::WARNING::2014-10-02 23:35:51,800::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 23:35:51,800::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 23:35:51,800::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 23:36:01,802::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 23:36:01,802::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107612::WARNING::2014-10-02 23:37:01,802::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 23:37:01,802::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 23:37:01,802::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 23:37:11,804::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 23:37:11,804::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107613::WARNING::2014-10-02 23:38:11,804::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 23:38:11,804::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 23:38:11,804::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-02 23:38:21,805::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-02 23:38:21,806::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 23:38:21,806::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107614::WARNING::2014-10-02 23:39:21,806::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 23:39:21,806::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 23:39:21,806::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 23:39:31,808::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 23:39:31,808::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107615::WARNING::2014-10-02 23:40:31,807::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 23:40:31,808::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 23:40:31,808::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 23:40:41,809::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 23:40:41,810::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107616::WARNING::2014-10-02 23:41:41,809::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 23:41:41,810::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 23:41:41,810::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 23:41:51,811::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 23:41:51,811::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107617::WARNING::2014-10-02 23:42:51,811::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 23:42:51,812::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 23:42:51,812::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 23:43:01,813::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 23:43:01,813::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107618::WARNING::2014-10-02 23:44:01,813::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 23:44:01,814::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 23:44:01,814::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-02 23:44:11,814::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-02 23:44:11,815::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 23:44:11,815::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107619::WARNING::2014-10-02 23:45:11,815::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 23:45:11,816::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 23:45:11,816::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 23:45:21,817::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 23:45:21,817::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107620::WARNING::2014-10-02 23:46:21,817::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 23:46:21,817::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 23:46:21,818::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 23:46:31,819::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 23:46:31,819::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107621::WARNING::2014-10-02 23:47:31,819::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 23:47:31,819::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 23:47:31,819::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 23:47:41,820::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 23:47:41,821::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107622::WARNING::2014-10-02 23:48:41,820::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 23:48:41,821::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 23:48:41,821::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 23:48:51,822::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 23:48:51,822::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107623::WARNING::2014-10-02 23:49:51,822::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 23:49:51,823::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 23:49:51,823::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-02 23:50:01,823::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-02 23:50:01,824::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 23:50:01,824::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107624::WARNING::2014-10-02 23:51:01,824::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 23:51:01,824::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 23:51:01,825::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 23:51:11,826::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 23:51:11,826::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107625::WARNING::2014-10-02 23:52:11,826::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 23:52:11,826::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 23:52:11,826::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 23:52:21,828::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 23:52:21,828::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107626::WARNING::2014-10-02 23:53:21,827::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 23:53:21,828::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 23:53:21,828::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 23:53:31,829::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 23:53:31,830::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107627::WARNING::2014-10-02 23:54:31,829::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 23:54:31,830::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 23:54:31,830::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 23:54:41,831::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 23:54:41,831::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107628::WARNING::2014-10-02 23:55:41,831::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 23:55:41,832::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 23:55:41,832::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-02 23:55:51,832::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-02 23:55:51,833::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 23:55:51,833::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107629::WARNING::2014-10-02 23:56:51,833::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 23:56:51,833::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 23:56:51,834::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 23:57:01,835::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 23:57:01,835::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107630::WARNING::2014-10-02 23:58:01,835::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 23:58:01,836::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 23:58:01,837::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 23:58:11,839::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 23:58:11,839::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107631::WARNING::2014-10-02 23:59:11,839::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-02 23:59:11,839::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-02 23:59:11,840::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-02 23:59:21,841::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-02 23:59:21,841::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107632::WARNING::2014-10-03 00:00:21,841::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 00:00:21,841::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 00:00:21,841::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 00:00:31,843::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 00:00:31,843::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107633::WARNING::2014-10-03 00:01:31,843::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 00:01:31,843::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 00:01:31,843::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 00:01:41,844::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 00:01:41,845::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 00:01:41,845::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107634::WARNING::2014-10-03 00:02:41,845::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 00:02:41,845::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 00:02:41,845::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 00:02:51,846::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 00:02:51,847::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107635::WARNING::2014-10-03 00:03:51,846::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 00:03:51,847::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 00:03:51,847::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 00:04:01,848::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 00:04:01,848::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107636::WARNING::2014-10-03 00:05:01,848::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 00:05:01,849::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 00:05:01,849::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 00:05:11,850::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 00:05:11,850::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107637::WARNING::2014-10-03 00:06:11,850::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 00:06:11,850::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 00:06:11,851::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 00:06:21,852::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 00:06:21,852::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107638::WARNING::2014-10-03 00:07:21,852::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 00:07:21,852::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 00:07:21,853::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 00:07:31,853::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 00:07:31,854::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 00:07:31,854::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107639::WARNING::2014-10-03 00:08:31,854::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 00:08:31,854::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 00:08:31,855::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 00:08:41,856::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 00:08:41,856::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107640::WARNING::2014-10-03 00:09:41,856::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 00:09:41,856::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 00:09:41,856::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 00:09:51,858::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 00:09:51,858::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107641::WARNING::2014-10-03 00:10:51,857::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 00:10:51,858::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 00:10:51,858::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 00:11:01,860::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 00:11:01,860::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107642::WARNING::2014-10-03 00:12:01,859::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 00:12:01,860::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 00:12:01,860::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 00:12:11,862::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 00:12:11,862::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107643::WARNING::2014-10-03 00:13:11,861::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 00:13:11,862::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 00:13:11,862::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 00:13:21,863::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 00:13:21,864::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 00:13:21,864::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107644::WARNING::2014-10-03 00:14:21,864::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 00:14:21,865::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 00:14:21,865::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 00:14:31,866::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 00:14:31,867::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107645::WARNING::2014-10-03 00:15:31,866::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 00:15:31,867::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 00:15:31,867::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 00:15:41,868::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 00:15:41,868::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107646::WARNING::2014-10-03 00:16:41,868::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 00:16:41,869::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 00:16:41,869::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 00:16:51,870::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 00:16:51,870::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107647::WARNING::2014-10-03 00:17:51,870::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 00:17:51,871::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 00:17:51,871::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 00:18:01,872::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 00:18:01,872::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107648::WARNING::2014-10-03 00:19:01,872::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 00:19:01,872::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 00:19:01,873::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 00:19:11,873::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 00:19:11,874::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 00:19:11,874::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107649::WARNING::2014-10-03 00:20:11,874::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 00:20:11,874::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 00:20:11,875::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 00:20:21,876::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 00:20:21,876::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107650::WARNING::2014-10-03 00:21:21,876::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 00:21:21,876::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 00:21:21,877::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 00:21:31,878::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 00:21:31,878::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107651::WARNING::2014-10-03 00:22:31,878::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 00:22:31,878::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 00:22:31,878::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 00:22:41,879::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 00:22:41,880::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107652::WARNING::2014-10-03 00:23:41,879::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 00:23:41,880::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 00:23:41,880::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 00:23:51,881::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 00:23:51,881::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107653::WARNING::2014-10-03 00:24:51,881::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 00:24:51,882::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 00:24:51,882::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 00:25:01,882::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 00:25:01,883::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 00:25:01,883::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107654::WARNING::2014-10-03 00:26:01,883::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 00:26:01,884::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 00:26:01,884::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 00:26:11,885::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 00:26:11,885::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107655::WARNING::2014-10-03 00:27:11,885::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 00:27:11,886::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 00:27:11,886::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 00:27:21,887::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 00:27:21,887::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107656::WARNING::2014-10-03 00:28:21,887::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 00:28:21,887::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 00:28:21,888::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 00:28:31,889::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 00:28:31,889::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107657::WARNING::2014-10-03 00:29:31,889::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 00:29:31,889::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 00:29:31,890::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 00:29:41,891::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 00:29:41,891::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107658::WARNING::2014-10-03 00:30:41,891::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 00:30:41,891::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 00:30:41,891::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 00:30:51,892::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 00:30:51,893::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 00:30:51,893::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107659::WARNING::2014-10-03 00:31:51,893::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 00:31:51,893::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 00:31:51,893::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 00:32:01,895::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 00:32:01,895::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107660::WARNING::2014-10-03 00:33:01,894::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 00:33:01,895::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 00:33:01,895::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 00:33:11,896::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 00:33:11,897::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-03 00:33:27,644::__init__::232::IOProcess::(_processLogs) (91964) Queuing response >Thread-18::DEBUG::2014-10-03 00:33:27,645::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 00:33:27,645::__init__::232::IOProcess::(_processLogs) (92024) Got request for method 'glob' >Thread-107661::WARNING::2014-10-03 00:34:11,896::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 00:34:11,897::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 00:34:11,897::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 00:34:21,898::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 00:34:21,898::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107662::WARNING::2014-10-03 00:35:21,898::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 00:35:21,899::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 00:35:21,899::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 00:35:31,900::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 00:35:31,900::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107663::WARNING::2014-10-03 00:36:31,900::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 00:36:31,900::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 00:36:31,901::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 00:36:41,901::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 00:36:41,902::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 00:36:41,902::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107664::WARNING::2014-10-03 00:37:41,902::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 00:37:41,902::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 00:37:41,903::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 00:37:51,904::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 00:37:51,904::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107665::WARNING::2014-10-03 00:38:51,904::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 00:38:51,904::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 00:38:51,905::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 00:39:01,906::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 00:39:01,906::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107666::WARNING::2014-10-03 00:40:01,906::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 00:40:01,906::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 00:40:01,907::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 00:40:11,908::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 00:40:11,908::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107667::WARNING::2014-10-03 00:41:11,908::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 00:41:11,908::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 00:41:11,909::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 00:41:21,910::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 00:41:21,910::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107668::WARNING::2014-10-03 00:42:21,910::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 00:42:21,910::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 00:42:21,911::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 00:42:31,911::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 00:42:31,912::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 00:42:31,912::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107669::WARNING::2014-10-03 00:43:31,912::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 00:43:31,912::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 00:43:31,913::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 00:43:41,914::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 00:43:41,914::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107670::WARNING::2014-10-03 00:44:41,914::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 00:44:41,914::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 00:44:41,914::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 00:44:51,917::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 00:44:51,917::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107671::WARNING::2014-10-03 00:45:51,916::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 00:45:51,916::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 00:45:51,917::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 00:46:01,918::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 00:46:01,918::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107672::WARNING::2014-10-03 00:47:01,918::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 00:47:01,918::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 00:47:01,918::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 00:47:11,919::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 00:47:11,920::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107673::WARNING::2014-10-03 00:48:11,919::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 00:48:11,920::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 00:48:11,920::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 00:48:21,920::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 00:48:21,921::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 00:48:21,922::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107674::WARNING::2014-10-03 00:49:21,921::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 00:49:21,922::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 00:49:21,922::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 00:49:31,923::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 00:49:31,923::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107675::WARNING::2014-10-03 00:50:31,923::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 00:50:31,924::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 00:50:31,924::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 00:50:41,925::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 00:50:41,925::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107676::WARNING::2014-10-03 00:51:41,925::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 00:51:41,925::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 00:51:41,926::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 00:51:51,927::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 00:51:51,927::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107677::WARNING::2014-10-03 00:52:51,927::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 00:52:51,927::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 00:52:51,927::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 00:53:01,930::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 00:53:01,931::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107678::WARNING::2014-10-03 00:54:01,930::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 00:54:01,930::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 00:54:01,930::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 00:54:11,931::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 00:54:11,932::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 00:54:11,932::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107679::WARNING::2014-10-03 00:55:11,932::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 00:55:11,932::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 00:55:11,932::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 00:55:21,934::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 00:55:21,934::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107680::WARNING::2014-10-03 00:56:21,934::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 00:56:21,934::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 00:56:21,934::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 00:56:31,935::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 00:56:31,936::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107681::WARNING::2014-10-03 00:57:31,935::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 00:57:31,936::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 00:57:31,936::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 00:57:41,937::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 00:57:41,937::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107682::WARNING::2014-10-03 00:58:41,937::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 00:58:41,938::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 00:58:41,938::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 00:58:51,939::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 00:58:51,939::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107683::WARNING::2014-10-03 00:59:51,939::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 00:59:51,940::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 00:59:51,940::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 01:00:01,940::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 01:00:01,941::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 01:00:01,941::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107684::WARNING::2014-10-03 01:01:01,941::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 01:01:01,942::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 01:01:01,942::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 01:01:11,943::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 01:01:11,943::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107685::WARNING::2014-10-03 01:02:11,943::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 01:02:11,944::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 01:02:11,944::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 01:02:21,945::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 01:02:21,945::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107686::WARNING::2014-10-03 01:03:21,945::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 01:03:21,945::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 01:03:21,946::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 01:03:31,947::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 01:03:31,947::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107687::WARNING::2014-10-03 01:04:31,947::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 01:04:31,947::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 01:04:31,948::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 01:04:41,949::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 01:04:41,949::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107688::WARNING::2014-10-03 01:05:41,949::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 01:05:41,949::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 01:05:41,950::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 01:05:51,950::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 01:05:51,951::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 01:05:51,951::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107689::WARNING::2014-10-03 01:06:51,951::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 01:06:51,951::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 01:06:51,951::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 01:07:01,953::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 01:07:01,953::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107690::WARNING::2014-10-03 01:08:01,953::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 01:08:01,953::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 01:08:01,953::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 01:08:11,954::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 01:08:11,955::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107691::WARNING::2014-10-03 01:09:11,954::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 01:09:11,955::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 01:09:11,955::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 01:09:21,956::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 01:09:21,956::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107692::WARNING::2014-10-03 01:10:21,956::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 01:10:21,957::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 01:10:21,957::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 01:10:31,958::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 01:10:31,958::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107693::WARNING::2014-10-03 01:11:31,958::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 01:11:31,959::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 01:11:31,959::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 01:11:41,959::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 01:11:41,960::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 01:11:41,960::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107694::WARNING::2014-10-03 01:12:41,960::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 01:12:41,961::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 01:12:41,961::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 01:12:51,963::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 01:12:51,963::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107695::WARNING::2014-10-03 01:13:51,963::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 01:13:51,963::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 01:13:51,963::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 01:14:01,965::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 01:14:01,965::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107696::WARNING::2014-10-03 01:15:01,965::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 01:15:01,965::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 01:15:01,965::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 01:15:11,967::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 01:15:11,967::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107697::WARNING::2014-10-03 01:16:11,967::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 01:16:11,968::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 01:16:11,968::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 01:16:21,969::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 01:16:21,969::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107698::WARNING::2014-10-03 01:17:21,969::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 01:17:21,969::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 01:17:21,970::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 01:17:31,970::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 01:17:31,971::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 01:17:31,971::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107699::WARNING::2014-10-03 01:18:31,971::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 01:18:31,971::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 01:18:31,972::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 01:18:41,973::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 01:18:41,973::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107700::WARNING::2014-10-03 01:19:41,973::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 01:19:41,973::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 01:19:41,974::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 01:19:51,975::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 01:19:51,975::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107701::WARNING::2014-10-03 01:20:51,975::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 01:20:51,975::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 01:20:51,975::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 01:21:01,977::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 01:21:01,977::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107702::WARNING::2014-10-03 01:22:01,976::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 01:22:01,977::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 01:22:01,977::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 01:22:11,978::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 01:22:11,978::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107703::WARNING::2014-10-03 01:23:11,978::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 01:23:11,979::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 01:23:11,979::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 01:23:21,979::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 01:23:21,980::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 01:23:21,980::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-03 01:23:48,445::__init__::232::IOProcess::(_processLogs) (91965) Queuing response >Thread-18::DEBUG::2014-10-03 01:23:48,445::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 01:23:48,445::__init__::232::IOProcess::(_processLogs) (92025) Got request for method 'glob' >Thread-107704::WARNING::2014-10-03 01:24:21,980::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 01:24:21,981::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 01:24:21,981::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 01:24:31,982::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 01:24:31,982::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107705::WARNING::2014-10-03 01:25:31,982::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 01:25:31,983::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 01:25:31,983::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 01:25:41,984::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 01:25:41,984::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107706::WARNING::2014-10-03 01:26:41,984::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 01:26:41,984::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 01:26:41,985::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 01:26:51,986::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 01:26:51,986::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107707::WARNING::2014-10-03 01:27:51,986::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 01:27:51,986::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 01:27:51,986::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 01:28:01,987::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 01:28:01,988::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107708::WARNING::2014-10-03 01:29:01,987::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 01:29:01,988::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 01:29:01,988::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 01:29:11,988::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 01:29:11,989::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 01:29:11,990::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107709::WARNING::2014-10-03 01:30:11,989::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 01:30:11,990::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 01:30:11,990::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 01:30:21,991::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 01:30:21,991::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107710::WARNING::2014-10-03 01:31:21,991::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 01:31:21,992::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 01:31:21,992::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 01:31:31,993::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 01:31:31,993::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107711::WARNING::2014-10-03 01:32:31,993::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 01:32:31,993::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 01:32:31,994::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 01:32:41,995::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 01:32:41,995::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107712::WARNING::2014-10-03 01:33:41,995::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 01:33:41,995::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 01:33:41,996::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 01:33:51,997::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 01:33:51,997::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-03 01:33:52,604::__init__::232::IOProcess::(_processLogs) (91966) Queuing response >Thread-18::DEBUG::2014-10-03 01:33:52,604::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 01:33:52,605::__init__::232::IOProcess::(_processLogs) (92026) Got request for method 'glob' >Thread-107713::WARNING::2014-10-03 01:34:51,997::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 01:34:51,997::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 01:34:51,997::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 01:35:01,998::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 01:35:01,999::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 01:35:01,999::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107714::WARNING::2014-10-03 01:36:01,999::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 01:36:01,999::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 01:36:01,999::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 01:36:12,000::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 01:36:12,001::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107715::WARNING::2014-10-03 01:37:12,000::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 01:37:12,001::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 01:37:12,001::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 01:37:22,002::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 01:37:22,003::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107716::WARNING::2014-10-03 01:38:22,002::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 01:38:22,003::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 01:38:22,003::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 01:38:32,004::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 01:38:32,005::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107717::WARNING::2014-10-03 01:39:32,004::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 01:39:32,005::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 01:39:32,005::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 01:39:42,006::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 01:39:42,007::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107718::WARNING::2014-10-03 01:40:42,006::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 01:40:42,007::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 01:40:42,007::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 01:40:52,007::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 01:40:52,008::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 01:40:52,009::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107719::WARNING::2014-10-03 01:41:52,008::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 01:41:52,009::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 01:41:52,009::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 01:42:02,010::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 01:42:02,011::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107720::WARNING::2014-10-03 01:43:02,010::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 01:43:02,011::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 01:43:02,011::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 01:43:12,012::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 01:43:12,013::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-03 01:43:56,764::__init__::232::IOProcess::(_processLogs) (91967) Queuing response >Thread-18::DEBUG::2014-10-03 01:43:56,765::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 01:43:56,765::__init__::232::IOProcess::(_processLogs) (92027) Got request for method 'glob' >Thread-107721::WARNING::2014-10-03 01:44:12,012::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 01:44:12,013::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 01:44:12,013::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 01:44:22,015::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 01:44:22,015::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107722::WARNING::2014-10-03 01:45:22,014::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 01:45:22,015::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 01:45:22,015::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 01:45:32,016::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 01:45:32,016::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107723::WARNING::2014-10-03 01:46:32,016::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 01:46:32,017::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 01:46:32,017::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 01:46:42,017::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 01:46:42,018::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 01:46:42,018::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107724::WARNING::2014-10-03 01:47:42,018::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 01:47:42,019::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 01:47:42,019::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 01:47:52,020::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 01:47:52,020::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107725::WARNING::2014-10-03 01:48:52,020::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 01:48:52,020::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 01:48:52,021::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 01:49:02,022::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 01:49:02,022::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107726::WARNING::2014-10-03 01:50:02,022::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 01:50:02,022::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 01:50:02,022::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 01:50:12,024::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 01:50:12,024::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107727::WARNING::2014-10-03 01:51:12,023::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 01:51:12,024::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 01:51:12,024::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 01:51:22,025::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 01:51:22,026::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107728::WARNING::2014-10-03 01:52:22,025::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 01:52:22,026::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 01:52:22,026::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 01:52:32,026::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 01:52:32,027::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 01:52:32,028::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107729::WARNING::2014-10-03 01:53:32,027::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 01:53:32,028::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 01:53:32,028::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 01:53:42,029::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 01:53:42,029::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107730::WARNING::2014-10-03 01:54:42,029::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 01:54:42,030::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 01:54:42,030::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 01:54:52,031::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 01:54:52,031::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107731::WARNING::2014-10-03 01:55:52,031::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 01:55:52,031::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 01:55:52,032::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 01:56:02,033::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 01:56:02,033::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107732::WARNING::2014-10-03 01:57:02,033::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 01:57:02,033::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 01:57:02,034::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 01:57:12,035::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 01:57:12,035::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107733::WARNING::2014-10-03 01:58:12,035::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 01:58:12,035::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 01:58:12,035::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 01:58:22,036::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 01:58:22,037::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 01:58:22,037::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107734::WARNING::2014-10-03 01:59:22,037::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 01:59:22,037::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 01:59:22,037::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 01:59:32,038::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 01:59:32,039::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107735::WARNING::2014-10-03 02:00:32,038::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 02:00:32,039::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 02:00:32,039::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 02:00:42,040::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 02:00:42,040::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107736::WARNING::2014-10-03 02:01:42,040::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 02:01:42,041::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 02:01:42,041::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 02:01:52,042::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 02:01:52,042::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107737::WARNING::2014-10-03 02:02:52,042::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 02:02:52,043::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 02:02:52,043::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 02:03:02,044::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 02:03:02,044::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107738::WARNING::2014-10-03 02:04:02,044::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 02:04:02,044::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 02:04:02,045::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 02:04:12,045::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 02:04:12,046::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 02:04:12,046::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107739::WARNING::2014-10-03 02:05:12,046::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 02:05:12,047::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 02:05:12,047::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 02:05:22,048::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 02:05:22,048::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107740::WARNING::2014-10-03 02:06:22,048::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 02:06:22,049::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 02:06:22,049::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 02:06:32,050::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 02:06:32,050::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107741::WARNING::2014-10-03 02:07:32,050::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 02:07:32,051::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 02:07:32,051::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 02:07:42,052::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 02:07:42,052::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107742::WARNING::2014-10-03 02:08:42,052::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 02:08:42,053::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 02:08:42,053::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 02:08:52,054::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 02:08:52,054::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107743::WARNING::2014-10-03 02:09:52,054::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 02:09:52,055::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 02:09:52,055::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 02:10:02,055::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 02:10:02,056::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 02:10:02,056::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107744::WARNING::2014-10-03 02:11:02,056::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 02:11:02,057::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 02:11:02,057::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 02:11:12,058::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 02:11:12,058::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107745::WARNING::2014-10-03 02:12:12,058::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 02:12:12,059::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 02:12:12,059::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 02:12:22,060::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 02:12:22,060::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107746::WARNING::2014-10-03 02:13:22,060::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 02:13:22,061::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 02:13:22,061::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 02:13:32,062::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 02:13:32,062::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107747::WARNING::2014-10-03 02:14:32,062::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 02:14:32,063::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 02:14:32,063::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 02:14:42,064::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 02:14:42,064::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107748::WARNING::2014-10-03 02:15:42,064::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 02:15:42,065::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 02:15:42,065::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 02:15:52,065::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 02:15:52,066::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 02:15:52,066::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107749::WARNING::2014-10-03 02:16:52,066::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 02:16:52,067::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 02:16:52,067::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 02:17:02,068::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 02:17:02,068::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107750::WARNING::2014-10-03 02:18:02,068::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 02:18:02,069::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 02:18:02,069::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 02:18:12,070::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 02:18:12,070::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107751::WARNING::2014-10-03 02:19:12,070::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 02:19:12,070::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 02:19:12,071::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 02:19:22,073::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 02:19:22,073::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107752::WARNING::2014-10-03 02:20:22,073::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 02:20:22,073::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 02:20:22,073::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 02:20:32,075::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 02:20:32,075::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107753::WARNING::2014-10-03 02:21:32,075::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 02:21:32,075::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 02:21:32,075::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 02:21:42,076::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 02:21:42,077::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 02:21:42,077::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107754::WARNING::2014-10-03 02:22:42,077::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 02:22:42,077::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 02:22:42,077::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 02:22:52,079::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 02:22:52,079::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107755::WARNING::2014-10-03 02:23:52,079::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 02:23:52,079::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 02:23:52,079::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 02:24:02,080::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 02:24:02,081::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-03 02:24:13,404::__init__::232::IOProcess::(_processLogs) (91968) Queuing response >Thread-18::DEBUG::2014-10-03 02:24:13,405::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 02:24:13,405::__init__::232::IOProcess::(_processLogs) (92028) Got request for method 'glob' >Thread-107756::WARNING::2014-10-03 02:25:02,080::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 02:25:02,081::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 02:25:02,081::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 02:25:12,082::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 02:25:12,083::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107757::WARNING::2014-10-03 02:26:12,082::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 02:26:12,083::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 02:26:12,083::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 02:26:22,084::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 02:26:22,084::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107758::WARNING::2014-10-03 02:27:22,084::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 02:27:22,085::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 02:27:22,085::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 02:27:32,085::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 02:27:32,086::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 02:27:32,086::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107759::WARNING::2014-10-03 02:28:32,086::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 02:28:32,087::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 02:28:32,087::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 02:28:42,088::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 02:28:42,088::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107760::WARNING::2014-10-03 02:29:42,088::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 02:29:42,089::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 02:29:42,089::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 02:29:52,090::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 02:29:52,090::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107761::WARNING::2014-10-03 02:30:52,090::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 02:30:52,091::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 02:30:52,091::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 02:31:02,092::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 02:31:02,092::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107762::WARNING::2014-10-03 02:32:02,092::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 02:32:02,093::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 02:32:02,093::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 02:32:12,094::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 02:32:12,094::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107763::WARNING::2014-10-03 02:33:12,094::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 02:33:12,095::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 02:33:12,095::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 02:33:22,095::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 02:33:22,096::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 02:33:22,096::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-03 02:34:17,564::__init__::232::IOProcess::(_processLogs) (91969) Queuing response >Thread-18::DEBUG::2014-10-03 02:34:17,565::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 02:34:17,565::__init__::232::IOProcess::(_processLogs) (92029) Got request for method 'glob' >Thread-107764::WARNING::2014-10-03 02:34:22,096::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 02:34:22,097::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 02:34:22,097::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 02:34:32,098::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 02:34:32,098::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107765::WARNING::2014-10-03 02:35:32,098::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 02:35:32,099::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 02:35:32,099::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 02:35:42,100::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 02:35:42,100::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107766::WARNING::2014-10-03 02:36:42,100::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 02:36:42,100::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 02:36:42,101::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 02:36:52,102::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 02:36:52,102::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107767::WARNING::2014-10-03 02:37:52,102::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 02:37:52,102::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 02:37:52,102::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 02:38:02,104::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 02:38:02,104::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107768::WARNING::2014-10-03 02:39:02,104::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 02:39:02,104::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 02:39:02,104::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 02:39:12,105::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 02:39:12,106::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 02:39:12,106::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107769::WARNING::2014-10-03 02:40:12,105::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 02:40:12,106::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 02:40:12,106::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 02:40:22,108::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 02:40:22,108::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107770::WARNING::2014-10-03 02:41:22,108::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 02:41:22,108::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 02:41:22,108::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 02:41:32,110::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 02:41:32,110::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107771::WARNING::2014-10-03 02:42:32,110::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 02:42:32,110::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 02:42:32,110::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 02:42:42,111::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 02:42:42,112::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107772::WARNING::2014-10-03 02:43:42,111::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 02:43:42,112::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 02:43:42,112::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 02:43:52,113::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 02:43:52,113::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107773::WARNING::2014-10-03 02:44:52,113::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 02:44:52,114::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 02:44:52,114::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 02:45:02,114::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 02:45:02,115::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 02:45:02,116::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107774::WARNING::2014-10-03 02:46:02,115::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 02:46:02,116::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 02:46:02,116::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 02:46:12,117::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 02:46:12,117::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107775::WARNING::2014-10-03 02:47:12,117::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 02:47:12,118::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 02:47:12,118::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 02:47:22,119::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 02:47:22,119::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107776::WARNING::2014-10-03 02:48:22,119::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 02:48:22,120::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 02:48:22,120::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 02:48:32,121::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 02:48:32,121::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107777::WARNING::2014-10-03 02:49:32,121::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 02:49:32,121::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 02:49:32,122::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 02:49:42,123::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 02:49:42,123::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107778::WARNING::2014-10-03 02:50:42,123::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 02:50:42,123::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 02:50:42,124::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 02:50:52,124::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 02:50:52,125::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 02:50:52,125::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107779::WARNING::2014-10-03 02:51:52,125::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 02:51:52,125::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 02:51:52,126::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 02:52:02,127::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 02:52:02,127::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107780::WARNING::2014-10-03 02:53:02,127::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 02:53:02,127::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 02:53:02,128::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 02:53:12,129::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 02:53:12,129::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107781::WARNING::2014-10-03 02:54:12,129::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 02:54:12,129::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 02:54:12,129::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 02:54:22,131::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 02:54:22,131::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107782::WARNING::2014-10-03 02:55:22,131::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 02:55:22,131::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 02:55:22,131::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 02:55:32,132::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 02:55:32,133::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107783::WARNING::2014-10-03 02:56:32,132::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 02:56:32,133::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 02:56:32,133::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 02:56:42,133::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 02:56:42,134::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 02:56:42,135::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107784::WARNING::2014-10-03 02:57:42,134::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 02:57:42,135::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 02:57:42,135::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 02:57:52,136::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 02:57:52,136::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107785::WARNING::2014-10-03 02:58:52,136::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 02:58:52,137::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 02:58:52,137::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 02:59:02,138::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 02:59:02,138::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107786::WARNING::2014-10-03 03:00:02,138::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 03:00:02,139::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 03:00:02,139::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 03:00:12,140::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 03:00:12,140::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107787::WARNING::2014-10-03 03:01:12,140::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 03:01:12,140::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 03:01:12,141::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 03:01:22,142::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 03:01:22,142::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107788::WARNING::2014-10-03 03:02:22,142::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 03:02:22,142::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 03:02:22,142::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 03:02:32,143::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 03:02:32,144::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 03:02:32,144::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107789::WARNING::2014-10-03 03:03:32,144::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 03:03:32,144::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 03:03:32,144::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 03:03:42,146::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 03:03:42,146::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107790::WARNING::2014-10-03 03:04:42,145::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 03:04:42,146::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 03:04:42,146::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 03:04:52,147::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 03:04:52,148::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107791::WARNING::2014-10-03 03:05:52,148::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 03:05:52,148::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 03:05:52,148::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 03:06:02,150::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 03:06:02,150::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107792::WARNING::2014-10-03 03:07:02,150::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 03:07:02,150::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 03:07:02,150::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 03:07:12,152::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 03:07:12,152::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107793::WARNING::2014-10-03 03:08:12,151::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 03:08:12,152::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 03:08:12,152::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 03:08:22,153::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 03:08:22,154::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 03:08:22,154::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107794::WARNING::2014-10-03 03:09:22,154::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 03:09:22,154::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 03:09:22,154::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 03:09:32,156::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 03:09:32,156::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107795::WARNING::2014-10-03 03:10:32,156::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 03:10:32,156::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 03:10:32,156::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 03:10:42,168::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 03:10:42,168::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107796::WARNING::2014-10-03 03:11:42,168::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 03:11:42,169::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 03:11:42,169::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 03:11:52,170::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 03:11:52,170::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107797::WARNING::2014-10-03 03:12:52,170::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 03:12:52,171::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 03:12:52,171::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 03:13:02,172::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 03:13:02,172::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107798::WARNING::2014-10-03 03:14:02,172::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 03:14:02,173::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 03:14:02,173::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 03:14:12,173::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 03:14:12,174::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 03:14:12,175::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107799::WARNING::2014-10-03 03:15:12,174::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 03:15:12,175::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 03:15:12,175::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 03:15:22,177::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 03:15:22,177::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107800::WARNING::2014-10-03 03:16:22,176::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 03:16:22,177::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 03:16:22,177::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 03:16:32,178::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 03:16:32,178::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107801::WARNING::2014-10-03 03:17:32,178::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 03:17:32,179::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 03:17:32,179::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 03:17:42,180::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 03:17:42,180::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107802::WARNING::2014-10-03 03:18:42,180::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 03:18:42,180::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 03:18:42,180::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 03:18:52,182::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 03:18:52,182::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107803::WARNING::2014-10-03 03:19:52,182::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 03:19:52,182::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 03:19:52,182::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 03:20:02,183::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 03:20:02,183::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 03:20:02,184::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107804::WARNING::2014-10-03 03:21:02,183::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 03:21:02,184::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 03:21:02,184::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 03:21:12,185::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 03:21:12,185::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107805::WARNING::2014-10-03 03:22:12,185::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 03:22:12,186::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 03:22:12,186::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 03:22:22,187::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 03:22:22,187::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107806::WARNING::2014-10-03 03:23:22,187::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 03:23:22,187::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 03:23:22,188::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 03:23:32,189::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 03:23:32,189::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107807::WARNING::2014-10-03 03:24:32,189::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 03:24:32,189::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 03:24:32,189::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 03:24:42,191::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 03:24:42,191::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107808::WARNING::2014-10-03 03:25:42,190::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 03:25:42,191::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 03:25:42,191::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 03:25:52,192::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 03:25:52,192::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 03:25:52,193::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107809::WARNING::2014-10-03 03:26:52,192::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 03:26:52,193::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 03:26:52,193::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 03:27:02,194::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 03:27:02,194::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107810::WARNING::2014-10-03 03:28:02,194::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 03:28:02,195::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 03:28:02,195::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 03:28:12,196::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 03:28:12,196::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107811::WARNING::2014-10-03 03:29:12,196::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 03:29:12,197::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 03:29:12,197::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 03:29:22,198::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 03:29:22,198::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107812::WARNING::2014-10-03 03:30:22,198::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 03:30:22,198::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 03:30:22,199::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 03:30:32,200::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 03:30:32,200::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107813::WARNING::2014-10-03 03:31:32,200::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 03:31:32,201::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 03:31:32,201::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 03:31:42,201::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 03:31:42,202::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 03:31:42,202::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107814::WARNING::2014-10-03 03:32:42,202::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 03:32:42,203::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 03:32:42,203::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 03:32:52,204::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 03:32:52,204::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107815::WARNING::2014-10-03 03:33:52,204::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 03:33:52,204::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 03:33:52,205::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 03:34:02,206::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 03:34:02,206::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107816::WARNING::2014-10-03 03:35:02,206::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 03:35:02,206::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 03:35:02,206::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 03:35:12,208::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 03:35:12,208::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107817::WARNING::2014-10-03 03:36:12,207::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 03:36:12,208::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 03:36:12,208::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 03:36:22,209::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 03:36:22,209::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107818::WARNING::2014-10-03 03:37:22,209::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 03:37:22,210::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 03:37:22,210::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 03:37:32,210::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 03:37:32,211::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 03:37:32,211::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107819::WARNING::2014-10-03 03:38:32,211::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 03:38:32,212::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 03:38:32,212::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 03:38:42,213::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 03:38:42,213::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107820::WARNING::2014-10-03 03:39:42,213::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 03:39:42,214::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 03:39:42,214::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 03:39:52,215::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 03:39:52,215::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107821::WARNING::2014-10-03 03:40:52,215::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 03:40:52,216::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 03:40:52,216::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 03:41:02,217::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 03:41:02,217::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107822::WARNING::2014-10-03 03:42:02,217::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 03:42:02,217::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 03:42:02,218::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 03:42:12,219::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 03:42:12,219::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107823::WARNING::2014-10-03 03:43:12,219::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 03:43:12,219::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 03:43:12,220::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 03:43:22,220::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 03:43:22,221::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 03:43:22,221::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107824::WARNING::2014-10-03 03:44:22,221::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 03:44:22,222::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 03:44:22,222::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 03:44:32,223::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 03:44:32,224::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107825::WARNING::2014-10-03 03:45:32,223::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 03:45:32,224::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 03:45:32,224::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 03:45:42,225::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 03:45:42,226::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107826::WARNING::2014-10-03 03:46:42,225::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 03:46:42,226::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 03:46:42,226::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 03:46:52,227::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 03:46:52,227::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107827::WARNING::2014-10-03 03:47:52,227::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 03:47:52,228::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 03:47:52,228::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 03:48:02,229::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 03:48:02,229::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107828::WARNING::2014-10-03 03:49:02,229::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 03:49:02,230::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 03:49:02,230::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 03:49:12,230::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 03:49:12,231::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 03:49:12,231::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107829::WARNING::2014-10-03 03:50:12,231::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 03:50:12,232::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 03:50:12,232::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 03:50:22,233::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 03:50:22,233::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107830::WARNING::2014-10-03 03:51:22,233::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 03:51:22,234::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 03:51:22,234::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 03:51:32,235::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 03:51:32,235::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107831::WARNING::2014-10-03 03:52:32,235::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 03:52:32,235::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 03:52:32,236::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 03:52:42,237::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 03:52:42,237::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107832::WARNING::2014-10-03 03:53:42,237::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 03:53:42,237::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 03:53:42,238::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 03:53:52,239::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 03:53:52,239::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107833::WARNING::2014-10-03 03:54:52,239::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 03:54:52,239::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 03:54:52,239::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 03:55:02,240::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 03:55:02,241::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 03:55:02,241::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107834::WARNING::2014-10-03 03:56:02,241::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 03:56:02,241::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 03:56:02,241::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 03:56:12,242::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 03:56:12,243::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107835::WARNING::2014-10-03 03:57:12,242::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 03:57:12,243::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 03:57:12,243::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 03:57:22,244::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 03:57:22,245::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107836::WARNING::2014-10-03 03:58:22,244::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 03:58:22,245::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 03:58:22,245::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 03:58:32,246::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 03:58:32,246::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107837::WARNING::2014-10-03 03:59:32,246::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 03:59:32,247::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 03:59:32,247::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 03:59:42,248::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 03:59:42,248::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107838::WARNING::2014-10-03 04:00:42,248::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 04:00:42,249::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 04:00:42,249::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 04:00:52,249::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 04:00:52,250::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 04:00:52,250::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107839::WARNING::2014-10-03 04:01:52,250::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 04:01:52,251::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 04:01:52,251::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 04:02:02,252::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 04:02:02,252::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107840::WARNING::2014-10-03 04:03:02,252::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 04:03:02,253::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 04:03:02,253::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 04:03:12,254::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 04:03:12,254::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107841::WARNING::2014-10-03 04:04:12,254::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 04:04:12,254::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 04:04:12,255::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 04:04:22,256::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 04:04:22,256::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-03 04:04:57,052::__init__::232::IOProcess::(_processLogs) (91970) Queuing response >Thread-18::DEBUG::2014-10-03 04:04:57,053::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 04:04:57,053::__init__::232::IOProcess::(_processLogs) (92030) Got request for method 'glob' >Thread-107842::WARNING::2014-10-03 04:05:22,256::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 04:05:22,256::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 04:05:22,256::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 04:05:32,258::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 04:05:32,258::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107843::WARNING::2014-10-03 04:06:32,258::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 04:06:32,258::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 04:06:32,258::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 04:06:42,259::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 04:06:42,260::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 04:06:42,260::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107844::WARNING::2014-10-03 04:07:42,260::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 04:07:42,260::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 04:07:42,260::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 04:07:52,261::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 04:07:52,262::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107845::WARNING::2014-10-03 04:08:52,261::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 04:08:52,262::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 04:08:52,262::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 04:09:02,263::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 04:09:02,264::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107846::WARNING::2014-10-03 04:10:02,263::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 04:10:02,264::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 04:10:02,264::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 04:10:12,265::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 04:10:12,266::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107847::WARNING::2014-10-03 04:11:12,265::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 04:11:12,266::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 04:11:12,266::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 04:11:22,267::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 04:11:22,268::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107848::WARNING::2014-10-03 04:12:22,267::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 04:12:22,268::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 04:12:22,268::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 04:12:32,268::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 04:12:32,270::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 04:12:32,270::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107849::WARNING::2014-10-03 04:13:32,269::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 04:13:32,270::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 04:13:32,270::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 04:13:42,272::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 04:13:42,272::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107850::WARNING::2014-10-03 04:14:42,271::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 04:14:42,272::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 04:14:42,272::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 04:14:52,273::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 04:14:52,274::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-03 04:15:01,212::__init__::232::IOProcess::(_processLogs) (91971) Queuing response >Thread-18::DEBUG::2014-10-03 04:15:01,213::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 04:15:01,213::__init__::232::IOProcess::(_processLogs) (92031) Got request for method 'glob' >Thread-107851::WARNING::2014-10-03 04:15:52,273::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 04:15:52,274::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 04:15:52,274::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 04:16:02,276::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 04:16:02,276::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107852::WARNING::2014-10-03 04:17:02,276::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 04:17:02,276::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 04:17:02,276::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 04:17:12,277::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 04:17:12,278::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107853::WARNING::2014-10-03 04:18:12,277::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 04:18:12,278::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 04:18:12,278::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 04:18:22,278::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 04:18:22,279::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 04:18:22,280::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107854::WARNING::2014-10-03 04:19:22,279::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 04:19:22,280::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 04:19:22,280::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 04:19:32,281::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 04:19:32,282::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107855::WARNING::2014-10-03 04:20:32,281::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 04:20:32,282::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 04:20:32,282::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 04:20:42,283::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 04:20:42,283::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107856::WARNING::2014-10-03 04:21:42,283::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 04:21:42,284::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 04:21:42,284::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 04:21:52,285::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 04:21:52,285::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107857::WARNING::2014-10-03 04:22:52,285::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 04:22:52,285::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 04:22:52,285::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 04:23:02,287::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 04:23:02,287::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107858::WARNING::2014-10-03 04:24:02,286::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 04:24:02,287::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 04:24:02,287::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 04:24:12,288::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 04:24:12,288::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 04:24:12,289::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-03 04:25:05,372::__init__::232::IOProcess::(_processLogs) (91972) Queuing response >Thread-18::DEBUG::2014-10-03 04:25:05,373::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 04:25:05,373::__init__::232::IOProcess::(_processLogs) (92032) Got request for method 'glob' >Thread-18::DEBUG::2014-10-03 04:25:05,373::__init__::232::IOProcess::(_processLogs) (91973) Queuing response >Thread-18::DEBUG::2014-10-03 04:25:05,373::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 04:25:05,373::__init__::232::IOProcess::(_processLogs) (92033) Got request for method 'glob' >Thread-107859::WARNING::2014-10-03 04:25:12,288::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 04:25:12,289::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 04:25:12,289::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 04:25:22,290::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 04:25:22,290::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107860::WARNING::2014-10-03 04:26:22,290::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 04:26:22,291::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 04:26:22,291::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 04:26:32,292::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 04:26:32,292::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107861::WARNING::2014-10-03 04:27:32,292::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 04:27:32,293::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 04:27:32,293::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 04:27:42,294::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 04:27:42,294::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107862::WARNING::2014-10-03 04:28:42,294::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 04:28:42,294::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 04:28:42,294::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 04:28:52,296::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 04:28:52,296::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107863::WARNING::2014-10-03 04:29:52,296::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 04:29:52,296::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 04:29:52,296::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 04:30:02,297::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 04:30:02,298::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 04:30:02,298::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107864::WARNING::2014-10-03 04:31:02,298::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 04:31:02,298::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 04:31:02,298::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 04:31:12,299::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 04:31:12,299::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107865::WARNING::2014-10-03 04:32:12,299::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 04:32:12,300::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 04:32:12,300::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 04:32:22,301::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 04:32:22,301::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107866::WARNING::2014-10-03 04:33:22,301::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 04:33:22,301::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 04:33:22,302::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 04:33:32,303::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 04:33:32,303::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107867::WARNING::2014-10-03 04:34:32,303::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 04:34:32,304::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 04:34:32,305::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 04:34:42,307::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 04:34:42,307::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-03 04:35:09,532::__init__::232::IOProcess::(_processLogs) (91974) Queuing response >Thread-18::DEBUG::2014-10-03 04:35:09,533::__init__::232::IOProcess::(_processLogs) (91975) Queuing response >Thread-18::DEBUG::2014-10-03 04:35:09,533::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 04:35:09,533::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 04:35:09,533::__init__::232::IOProcess::(_processLogs) (92035) Got request for method 'glob' >Thread-18::DEBUG::2014-10-03 04:35:09,533::__init__::232::IOProcess::(_processLogs) (92034) Got request for method 'glob' >Thread-107868::WARNING::2014-10-03 04:35:42,307::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 04:35:42,307::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 04:35:42,308::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 04:35:52,308::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 04:35:52,309::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 04:35:52,309::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107869::WARNING::2014-10-03 04:36:52,309::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 04:36:52,309::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 04:36:52,310::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 04:37:02,311::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 04:37:02,311::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107870::WARNING::2014-10-03 04:38:02,311::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 04:38:02,311::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 04:38:02,312::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 04:38:12,313::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 04:38:12,313::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107871::WARNING::2014-10-03 04:39:12,313::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 04:39:12,313::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 04:39:12,313::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 04:39:22,315::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 04:39:22,315::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107872::WARNING::2014-10-03 04:40:22,315::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 04:40:22,315::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 04:40:22,315::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 04:40:32,317::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 04:40:32,317::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107873::WARNING::2014-10-03 04:41:32,316::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 04:41:32,317::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 04:41:32,317::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 04:41:42,318::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 04:41:42,319::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 04:41:42,319::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107874::WARNING::2014-10-03 04:42:42,319::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 04:42:42,319::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 04:42:42,319::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 04:42:52,321::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 04:42:52,321::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107875::WARNING::2014-10-03 04:43:52,321::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 04:43:52,321::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 04:43:52,321::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 04:44:02,323::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 04:44:02,323::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107876::WARNING::2014-10-03 04:45:02,323::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 04:45:02,323::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 04:45:02,323::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 04:45:12,325::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 04:45:12,325::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-03 04:45:13,692::__init__::232::IOProcess::(_processLogs) (91976) Queuing response >Thread-18::DEBUG::2014-10-03 04:45:13,693::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 04:45:13,693::__init__::232::IOProcess::(_processLogs) (92036) Got request for method 'glob' >Thread-18::DEBUG::2014-10-03 04:45:13,693::__init__::232::IOProcess::(_processLogs) (91978) Queuing response >Thread-18::DEBUG::2014-10-03 04:45:13,693::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 04:45:13,693::__init__::232::IOProcess::(_processLogs) (92037) Got request for method 'glob' >Thread-107877::WARNING::2014-10-03 04:46:12,325::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 04:46:12,325::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 04:46:12,325::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 04:46:22,327::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 04:46:22,327::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107878::WARNING::2014-10-03 04:47:22,326::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 04:47:22,327::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 04:47:22,327::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 04:47:32,328::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 04:47:32,329::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 04:47:32,329::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107879::WARNING::2014-10-03 04:48:32,329::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 04:48:32,329::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 04:48:32,329::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 04:48:42,330::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 04:48:42,331::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107880::WARNING::2014-10-03 04:49:42,330::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 04:49:42,331::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 04:49:42,331::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 04:49:52,332::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 04:49:52,332::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107881::WARNING::2014-10-03 04:50:52,332::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 04:50:52,333::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 04:50:52,333::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 04:51:02,334::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 04:51:02,334::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107882::WARNING::2014-10-03 04:52:02,334::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 04:52:02,335::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 04:52:02,335::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 04:52:12,336::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 04:52:12,336::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107883::WARNING::2014-10-03 04:53:12,336::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 04:53:12,337::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 04:53:12,337::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 04:53:22,337::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 04:53:22,338::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 04:53:22,338::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107884::WARNING::2014-10-03 04:54:22,338::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 04:54:22,339::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 04:54:22,339::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 04:54:32,340::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 04:54:32,341::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-03 04:55:17,852::__init__::232::IOProcess::(_processLogs) (91977) Queuing response >Thread-18::DEBUG::2014-10-03 04:55:17,853::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 04:55:17,853::__init__::232::IOProcess::(_processLogs) (92038) Got request for method 'glob' >Thread-18::DEBUG::2014-10-03 04:55:17,853::__init__::232::IOProcess::(_processLogs) (91979) Queuing response >Thread-18::DEBUG::2014-10-03 04:55:17,853::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 04:55:17,853::__init__::232::IOProcess::(_processLogs) (92039) Got request for method 'glob' >Thread-107885::WARNING::2014-10-03 04:55:32,340::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 04:55:32,341::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 04:55:32,341::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 04:55:42,342::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 04:55:42,342::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107886::WARNING::2014-10-03 04:56:42,342::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 04:56:42,343::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 04:56:42,343::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 04:56:52,344::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 04:56:52,344::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107887::WARNING::2014-10-03 04:57:52,344::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 04:57:52,345::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 04:57:52,345::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 04:58:02,346::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 04:58:02,346::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107888::WARNING::2014-10-03 04:59:02,346::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 04:59:02,347::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 04:59:02,347::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 04:59:12,348::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 04:59:12,348::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 04:59:12,349::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107889::WARNING::2014-10-03 05:00:12,348::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 05:00:12,349::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 05:00:12,349::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 05:00:22,350::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 05:00:22,350::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107890::WARNING::2014-10-03 05:01:22,350::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 05:01:22,351::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 05:01:22,351::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 05:01:32,352::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 05:01:32,352::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107891::WARNING::2014-10-03 05:02:32,352::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 05:02:32,353::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 05:02:32,353::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 05:02:42,354::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 05:02:42,354::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107892::WARNING::2014-10-03 05:03:42,354::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 05:03:42,355::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 05:03:42,355::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 05:03:52,356::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 05:03:52,356::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107893::WARNING::2014-10-03 05:04:52,356::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 05:04:52,356::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 05:04:52,357::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 05:05:02,357::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 05:05:02,358::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 05:05:02,358::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-03 05:05:22,012::__init__::232::IOProcess::(_processLogs) (91980) Queuing response >Thread-18::DEBUG::2014-10-03 05:05:22,013::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 05:05:22,013::__init__::232::IOProcess::(_processLogs) (92040) Got request for method 'glob' >Thread-18::DEBUG::2014-10-03 05:05:22,013::__init__::232::IOProcess::(_processLogs) (91981) Queuing response >Thread-18::DEBUG::2014-10-03 05:05:22,013::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 05:05:22,013::__init__::232::IOProcess::(_processLogs) (92041) Got request for method 'glob' >Thread-107894::WARNING::2014-10-03 05:06:02,358::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 05:06:02,358::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 05:06:02,359::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 05:06:12,360::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 05:06:12,360::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107895::WARNING::2014-10-03 05:07:12,360::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 05:07:12,360::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 05:07:12,360::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 05:07:22,362::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 05:07:22,362::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107896::WARNING::2014-10-03 05:08:22,362::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 05:08:22,362::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 05:08:22,362::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 05:08:32,364::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 05:08:32,364::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107897::WARNING::2014-10-03 05:09:32,363::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 05:09:32,364::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 05:09:32,364::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 05:09:42,365::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 05:09:42,366::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107898::WARNING::2014-10-03 05:10:42,365::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 05:10:42,366::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 05:10:42,366::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 05:10:52,367::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 05:10:52,368::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 05:10:52,368::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107899::WARNING::2014-10-03 05:11:52,368::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 05:11:52,368::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 05:11:52,368::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 05:12:02,369::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 05:12:02,370::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107900::WARNING::2014-10-03 05:13:02,369::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 05:13:02,370::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 05:13:02,370::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 05:13:12,371::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 05:13:12,372::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107901::WARNING::2014-10-03 05:14:12,371::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 05:14:12,372::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 05:14:12,372::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 05:14:22,373::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 05:14:22,374::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107902::WARNING::2014-10-03 05:15:22,373::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 05:15:22,374::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 05:15:22,374::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 05:15:26,172::__init__::232::IOProcess::(_processLogs) (91983) Queuing response >Thread-18::DEBUG::2014-10-03 05:15:26,172::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 05:15:26,173::__init__::232::IOProcess::(_processLogs) (92042) Got request for method 'glob' >Thread-18::DEBUG::2014-10-03 05:15:26,173::__init__::232::IOProcess::(_processLogs) (91982) Queuing response >Thread-18::DEBUG::2014-10-03 05:15:26,173::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 05:15:26,173::__init__::232::IOProcess::(_processLogs) (92043) Got request for method 'glob' >Thread-18::DEBUG::2014-10-03 05:15:32,375::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 05:15:32,376::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107903::WARNING::2014-10-03 05:16:32,375::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 05:16:32,376::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 05:16:32,376::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 05:16:42,376::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 05:16:42,377::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 05:16:42,378::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107904::WARNING::2014-10-03 05:17:42,377::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 05:17:42,378::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 05:17:42,378::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 05:17:52,379::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 05:17:52,379::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107905::WARNING::2014-10-03 05:18:52,379::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 05:18:52,379::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 05:18:52,380::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 05:19:02,381::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 05:19:02,381::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107906::WARNING::2014-10-03 05:20:02,381::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 05:20:02,381::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 05:20:02,381::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 05:20:12,383::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 05:20:12,383::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107907::WARNING::2014-10-03 05:21:12,383::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 05:21:12,383::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 05:21:12,383::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 05:21:22,384::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 05:21:22,385::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107908::WARNING::2014-10-03 05:22:22,384::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 05:22:22,385::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 05:22:22,385::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 05:22:32,385::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 05:22:32,386::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 05:22:32,386::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107909::WARNING::2014-10-03 05:23:32,386::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 05:23:32,387::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 05:23:32,387::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 05:23:42,388::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 05:23:42,388::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107910::WARNING::2014-10-03 05:24:42,388::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 05:24:42,388::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 05:24:42,389::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 05:24:52,390::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 05:24:52,390::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-03 05:25:30,332::__init__::232::IOProcess::(_processLogs) (91984) Queuing response >Thread-18::DEBUG::2014-10-03 05:25:30,333::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 05:25:30,333::__init__::232::IOProcess::(_processLogs) (92044) Got request for method 'glob' >Thread-18::DEBUG::2014-10-03 05:25:30,333::__init__::232::IOProcess::(_processLogs) (91985) Queuing response >Thread-18::DEBUG::2014-10-03 05:25:30,333::__init__::232::IOProcess::(_processLogs) (91986) Queuing response >Thread-18::DEBUG::2014-10-03 05:25:30,333::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 05:25:30,333::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 05:25:30,333::__init__::232::IOProcess::(_processLogs) (92045) Got request for method 'glob' >Thread-18::DEBUG::2014-10-03 05:25:30,333::__init__::232::IOProcess::(_processLogs) (92046) Got request for method 'glob' >Thread-107911::WARNING::2014-10-03 05:25:52,390::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 05:25:52,390::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 05:25:52,390::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 05:26:02,391::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 05:26:02,392::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107912::WARNING::2014-10-03 05:27:02,391::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 05:27:02,392::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 05:27:02,392::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 05:27:12,393::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 05:27:12,393::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107913::WARNING::2014-10-03 05:28:12,393::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 05:28:12,394::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 05:28:12,394::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 05:28:22,394::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 05:28:22,395::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 05:28:22,395::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107914::WARNING::2014-10-03 05:29:22,395::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 05:29:22,395::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 05:29:22,396::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 05:29:32,397::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 05:29:32,397::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107915::WARNING::2014-10-03 05:30:32,397::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 05:30:32,397::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 05:30:32,398::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 05:30:42,399::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 05:30:42,399::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107916::WARNING::2014-10-03 05:31:42,399::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 05:31:42,399::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 05:31:42,399::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 05:31:52,401::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 05:31:52,401::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107917::WARNING::2014-10-03 05:32:52,400::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 05:32:52,401::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 05:32:52,401::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 05:33:02,402::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 05:33:02,403::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107918::WARNING::2014-10-03 05:34:02,402::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 05:34:02,403::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 05:34:02,403::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 05:34:12,403::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 05:34:12,404::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 05:34:12,405::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107919::WARNING::2014-10-03 05:35:12,404::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 05:35:12,405::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 05:35:12,405::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 05:35:22,406::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 05:35:22,406::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-03 05:35:34,492::__init__::232::IOProcess::(_processLogs) (91987) Queuing response >Thread-18::DEBUG::2014-10-03 05:35:34,493::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 05:35:34,493::__init__::232::IOProcess::(_processLogs) (92047) Got request for method 'glob' >Thread-107920::WARNING::2014-10-03 05:36:22,406::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 05:36:22,407::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 05:36:22,407::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 05:36:32,408::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 05:36:32,408::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107921::WARNING::2014-10-03 05:37:32,408::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 05:37:32,409::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 05:37:32,409::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 05:37:42,410::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 05:37:42,410::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107922::WARNING::2014-10-03 05:38:42,410::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 05:38:42,410::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 05:38:42,411::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 05:38:52,412::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 05:38:52,412::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107923::WARNING::2014-10-03 05:39:52,412::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 05:39:52,412::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 05:39:52,412::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 05:40:02,413::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 05:40:02,414::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 05:40:02,414::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107924::WARNING::2014-10-03 05:41:02,414::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 05:41:02,414::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 05:41:02,414::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 05:41:12,416::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 05:41:12,416::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107925::WARNING::2014-10-03 05:42:12,416::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 05:42:12,416::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 05:42:12,416::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 05:42:22,417::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 05:42:22,418::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107926::WARNING::2014-10-03 05:43:22,417::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 05:43:22,418::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 05:43:22,418::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 05:43:32,419::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 05:43:32,419::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107927::WARNING::2014-10-03 05:44:32,419::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 05:44:32,420::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 05:44:32,420::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 05:44:42,421::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 05:44:42,421::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-03 05:45:38,652::__init__::232::IOProcess::(_processLogs) (91988) Queuing response >Thread-18::DEBUG::2014-10-03 05:45:38,653::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 05:45:38,653::__init__::232::IOProcess::(_processLogs) (92048) Got request for method 'glob' >Thread-18::DEBUG::2014-10-03 05:45:38,653::__init__::232::IOProcess::(_processLogs) (91989) Queuing response >Thread-18::DEBUG::2014-10-03 05:45:38,653::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 05:45:38,653::__init__::232::IOProcess::(_processLogs) (92049) Got request for method 'glob' >Thread-107928::WARNING::2014-10-03 05:45:42,421::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 05:45:42,422::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 05:45:42,422::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 05:45:52,422::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 05:45:52,423::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 05:45:52,424::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107929::WARNING::2014-10-03 05:46:52,423::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 05:46:52,424::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 05:46:52,424::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 05:47:02,425::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 05:47:02,426::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107930::WARNING::2014-10-03 05:48:02,425::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 05:48:02,426::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 05:48:02,426::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 05:48:12,427::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 05:48:12,427::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107931::WARNING::2014-10-03 05:49:12,427::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 05:49:12,428::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 05:49:12,428::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 05:49:22,429::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 05:49:22,429::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107932::WARNING::2014-10-03 05:50:22,429::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 05:50:22,429::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 05:50:22,430::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 05:50:32,431::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 05:50:32,431::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107933::WARNING::2014-10-03 05:51:32,431::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 05:51:32,431::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 05:51:32,432::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 05:51:42,432::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 05:51:42,433::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 05:51:42,433::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107934::WARNING::2014-10-03 05:52:42,433::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 05:52:42,433::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 05:52:42,434::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 05:52:52,435::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 05:52:52,435::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107935::WARNING::2014-10-03 05:53:52,435::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 05:53:52,435::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 05:53:52,435::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 05:54:02,437::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 05:54:02,437::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107936::WARNING::2014-10-03 05:55:02,436::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 05:55:02,437::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 05:55:02,437::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 05:55:12,438::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 05:55:12,439::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-03 05:55:42,812::__init__::232::IOProcess::(_processLogs) (91990) Queuing response >Thread-18::DEBUG::2014-10-03 05:55:42,813::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 05:55:42,813::__init__::232::IOProcess::(_processLogs) (92050) Got request for method 'glob' >Thread-18::DEBUG::2014-10-03 05:55:42,813::__init__::232::IOProcess::(_processLogs) (91991) Queuing response >Thread-18::DEBUG::2014-10-03 05:55:42,813::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 05:55:42,813::__init__::232::IOProcess::(_processLogs) (92051) Got request for method 'glob' >Thread-107937::WARNING::2014-10-03 05:56:12,438::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 05:56:12,439::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 05:56:12,439::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 05:56:22,440::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 05:56:22,440::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107938::WARNING::2014-10-03 05:57:22,440::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 05:57:22,441::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 05:57:22,441::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 05:57:32,441::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 05:57:32,442::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 05:57:32,442::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107939::WARNING::2014-10-03 05:58:32,442::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 05:58:32,443::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 05:58:32,443::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 05:58:42,444::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 05:58:42,444::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107940::WARNING::2014-10-03 05:59:42,444::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 05:59:42,444::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 05:59:42,445::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 05:59:52,446::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 05:59:52,446::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107941::WARNING::2014-10-03 06:00:52,446::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 06:00:52,446::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 06:00:52,446::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 06:01:02,448::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 06:01:02,448::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107942::WARNING::2014-10-03 06:02:02,448::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 06:02:02,448::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 06:02:02,448::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 06:02:12,449::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 06:02:12,450::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107943::WARNING::2014-10-03 06:03:12,449::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 06:03:12,450::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 06:03:12,450::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 06:03:22,450::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 06:03:22,451::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 06:03:22,452::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107944::WARNING::2014-10-03 06:04:22,451::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 06:04:22,452::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 06:04:22,452::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 06:04:32,453::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 06:04:32,453::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107945::WARNING::2014-10-03 06:05:32,453::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 06:05:32,454::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 06:05:32,454::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 06:05:42,455::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 06:05:42,455::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-03 06:05:46,972::__init__::232::IOProcess::(_processLogs) (91992) Queuing response >Thread-18::DEBUG::2014-10-03 06:05:46,973::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 06:05:46,973::__init__::232::IOProcess::(_processLogs) (92052) Got request for method 'glob' >Thread-18::DEBUG::2014-10-03 06:05:46,973::__init__::232::IOProcess::(_processLogs) (91993) Queuing response >Thread-18::DEBUG::2014-10-03 06:05:46,973::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 06:05:46,973::__init__::232::IOProcess::(_processLogs) (92053) Got request for method 'glob' >Thread-107946::WARNING::2014-10-03 06:06:42,455::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 06:06:42,456::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 06:06:42,456::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 06:06:52,457::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 06:06:52,457::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107947::WARNING::2014-10-03 06:07:52,457::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 06:07:52,457::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 06:07:52,458::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 06:08:02,459::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 06:08:02,459::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107948::WARNING::2014-10-03 06:09:02,459::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 06:09:02,459::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 06:09:02,459::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 06:09:12,460::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 06:09:12,461::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 06:09:12,461::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107949::WARNING::2014-10-03 06:10:12,461::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 06:10:12,461::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 06:10:12,461::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 06:10:22,463::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 06:10:22,463::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107950::WARNING::2014-10-03 06:11:22,463::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 06:11:22,463::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 06:11:22,463::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 06:11:32,464::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 06:11:32,465::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107951::WARNING::2014-10-03 06:12:32,464::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 06:12:32,465::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 06:12:32,465::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 06:12:42,466::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 06:12:42,467::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107952::WARNING::2014-10-03 06:13:42,466::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 06:13:42,467::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 06:13:42,467::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 06:13:52,468::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 06:13:52,468::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107953::WARNING::2014-10-03 06:14:52,468::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 06:14:52,469::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 06:14:52,469::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 06:15:02,469::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 06:15:02,470::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 06:15:02,470::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-03 06:15:51,132::__init__::232::IOProcess::(_processLogs) (91994) Queuing response >Thread-18::DEBUG::2014-10-03 06:15:51,133::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 06:15:51,133::__init__::232::IOProcess::(_processLogs) (92054) Got request for method 'glob' >Thread-18::DEBUG::2014-10-03 06:15:51,133::__init__::232::IOProcess::(_processLogs) (91995) Queuing response >Thread-18::DEBUG::2014-10-03 06:15:51,133::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 06:15:51,133::__init__::232::IOProcess::(_processLogs) (92055) Got request for method 'glob' >Thread-107954::WARNING::2014-10-03 06:16:02,470::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 06:16:02,471::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 06:16:02,471::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 06:16:12,474::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 06:16:12,474::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107955::WARNING::2014-10-03 06:17:12,473::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 06:17:12,474::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 06:17:12,474::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 06:17:22,475::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 06:17:22,475::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107956::WARNING::2014-10-03 06:18:22,475::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 06:18:22,475::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 06:18:22,476::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 06:18:32,477::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 06:18:32,477::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107957::WARNING::2014-10-03 06:19:32,477::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 06:19:32,477::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 06:19:32,477::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 06:19:42,478::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 06:19:42,479::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107958::WARNING::2014-10-03 06:20:42,478::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 06:20:42,479::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 06:20:42,479::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 06:20:52,479::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 06:20:52,480::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 06:20:52,480::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107959::WARNING::2014-10-03 06:21:52,480::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 06:21:52,481::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 06:21:52,481::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 06:22:02,482::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 06:22:02,482::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107960::WARNING::2014-10-03 06:23:02,482::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 06:23:02,482::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 06:23:02,483::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 06:23:12,484::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 06:23:12,484::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107961::WARNING::2014-10-03 06:24:12,484::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 06:24:12,484::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 06:24:12,484::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 06:24:22,486::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 06:24:22,486::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107962::WARNING::2014-10-03 06:25:22,485::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 06:25:22,486::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 06:25:22,486::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 06:25:32,487::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 06:25:32,487::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-03 06:25:55,292::__init__::232::IOProcess::(_processLogs) (91996) Queuing response >Thread-18::DEBUG::2014-10-03 06:25:55,293::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 06:25:55,293::__init__::232::IOProcess::(_processLogs) (92056) Got request for method 'glob' >Thread-107963::WARNING::2014-10-03 06:26:32,487::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 06:26:32,488::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 06:26:32,488::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 06:26:42,488::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 06:26:42,489::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 06:26:42,489::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107964::WARNING::2014-10-03 06:27:42,489::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 06:27:42,490::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 06:27:42,490::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 06:27:52,491::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 06:27:52,491::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107965::WARNING::2014-10-03 06:28:52,491::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 06:28:52,492::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 06:28:52,492::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 06:29:02,493::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 06:29:02,493::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107966::WARNING::2014-10-03 06:30:02,493::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 06:30:02,494::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 06:30:02,494::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 06:30:12,495::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 06:30:12,495::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107967::WARNING::2014-10-03 06:31:12,495::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 06:31:12,495::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 06:31:12,496::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 06:31:22,497::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 06:31:22,497::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107968::WARNING::2014-10-03 06:32:22,497::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 06:32:22,497::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 06:32:22,498::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 06:32:32,498::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 06:32:32,499::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 06:32:32,499::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107969::WARNING::2014-10-03 06:33:32,499::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 06:33:32,499::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 06:33:32,500::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 06:33:42,501::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 06:33:42,501::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107970::WARNING::2014-10-03 06:34:42,501::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 06:34:42,501::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 06:34:42,501::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 06:34:52,503::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 06:34:52,503::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107971::WARNING::2014-10-03 06:35:52,503::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 06:35:52,503::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 06:35:52,503::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 06:35:59,453::__init__::232::IOProcess::(_processLogs) (91997) Queuing response >Thread-18::DEBUG::2014-10-03 06:35:59,453::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 06:35:59,453::__init__::232::IOProcess::(_processLogs) (92057) Got request for method 'glob' >Thread-18::DEBUG::2014-10-03 06:36:02,504::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 06:36:02,505::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107972::WARNING::2014-10-03 06:37:02,504::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 06:37:02,505::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 06:37:02,505::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 06:37:12,506::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 06:37:12,506::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107973::WARNING::2014-10-03 06:38:12,506::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 06:38:12,507::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 06:38:12,507::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 06:38:22,507::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 06:38:22,508::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 06:38:22,508::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107974::WARNING::2014-10-03 06:39:22,508::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 06:39:22,509::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 06:39:22,509::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 06:39:32,510::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 06:39:32,510::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107975::WARNING::2014-10-03 06:40:32,510::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 06:40:32,511::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 06:40:32,511::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 06:40:42,512::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 06:40:42,512::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107976::WARNING::2014-10-03 06:41:42,512::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 06:41:42,513::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 06:41:42,513::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 06:41:52,514::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 06:41:52,514::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107977::WARNING::2014-10-03 06:42:52,514::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 06:42:52,514::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 06:42:52,515::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 06:43:02,516::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 06:43:02,516::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107978::WARNING::2014-10-03 06:44:02,516::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 06:44:02,516::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 06:44:02,517::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 06:44:12,517::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 06:44:12,518::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 06:44:12,518::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107979::WARNING::2014-10-03 06:45:12,518::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 06:45:12,518::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 06:45:12,519::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 06:45:22,520::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 06:45:22,520::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-03 06:46:03,612::__init__::232::IOProcess::(_processLogs) (91998) Queuing response >Thread-18::DEBUG::2014-10-03 06:46:03,613::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 06:46:03,613::__init__::232::IOProcess::(_processLogs) (92058) Got request for method 'glob' >Thread-107980::WARNING::2014-10-03 06:46:22,520::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 06:46:22,520::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 06:46:22,521::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 06:46:32,522::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 06:46:32,522::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107981::WARNING::2014-10-03 06:47:32,522::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 06:47:32,522::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 06:47:32,522::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 06:47:42,524::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 06:47:42,524::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107982::WARNING::2014-10-03 06:48:42,524::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 06:48:42,524::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 06:48:42,524::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 06:48:52,525::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 06:48:52,526::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107983::WARNING::2014-10-03 06:49:52,525::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 06:49:52,526::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 06:49:52,526::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 06:50:02,527::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 06:50:02,528::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 06:50:02,528::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107984::WARNING::2014-10-03 06:51:02,528::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 06:51:02,528::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 06:51:02,528::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 06:51:12,529::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 06:51:12,530::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107985::WARNING::2014-10-03 06:52:12,529::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 06:52:12,530::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 06:52:12,530::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 06:52:22,531::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 06:52:22,531::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107986::WARNING::2014-10-03 06:53:22,531::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 06:53:22,532::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 06:53:22,532::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 06:53:32,533::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 06:53:32,533::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107987::WARNING::2014-10-03 06:54:32,533::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 06:54:32,534::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 06:54:32,534::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 06:54:42,535::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 06:54:42,535::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107988::WARNING::2014-10-03 06:55:42,535::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 06:55:42,536::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 06:55:42,536::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 06:55:52,536::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 06:55:52,537::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 06:55:52,537::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-03 06:56:07,772::__init__::232::IOProcess::(_processLogs) (91999) Queuing response >Thread-18::DEBUG::2014-10-03 06:56:07,773::__init__::232::IOProcess::(_processLogs) (92000) Queuing response >Thread-18::DEBUG::2014-10-03 06:56:07,774::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 06:56:07,774::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 06:56:07,775::__init__::232::IOProcess::(_processLogs) (92059) Got request for method 'glob' >Thread-18::DEBUG::2014-10-03 06:56:07,775::__init__::232::IOProcess::(_processLogs) (92060) Got request for method 'glob' >Thread-107989::WARNING::2014-10-03 06:56:52,537::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 06:56:52,538::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 06:56:52,538::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 06:57:02,539::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 06:57:02,539::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107990::WARNING::2014-10-03 06:58:02,539::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 06:58:02,539::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 06:58:02,539::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 06:58:12,541::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 06:58:12,541::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107991::WARNING::2014-10-03 06:59:12,541::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 06:59:12,541::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 06:59:12,541::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 06:59:22,543::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 06:59:22,543::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107992::WARNING::2014-10-03 07:00:22,542::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 07:00:22,543::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 07:00:22,543::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 07:00:32,544::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 07:00:32,544::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107993::WARNING::2014-10-03 07:01:32,544::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 07:01:32,545::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 07:01:32,545::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 07:01:42,545::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 07:01:42,546::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 07:01:42,547::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107994::WARNING::2014-10-03 07:02:42,546::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 07:02:42,547::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 07:02:42,547::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 07:02:52,548::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 07:02:52,548::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107995::WARNING::2014-10-03 07:03:52,548::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 07:03:52,549::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 07:03:52,549::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 07:04:02,550::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 07:04:02,550::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107996::WARNING::2014-10-03 07:05:02,550::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 07:05:02,550::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 07:05:02,551::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 07:05:12,552::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 07:05:12,552::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107997::WARNING::2014-10-03 07:06:12,552::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 07:06:12,552::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 07:06:12,553::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 07:06:13,980::__init__::232::IOProcess::(_processLogs) (92001) Queuing response >Thread-18::DEBUG::2014-10-03 07:06:13,980::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 07:06:13,981::__init__::232::IOProcess::(_processLogs) (92061) Got request for method 'glob' >Thread-18::DEBUG::2014-10-03 07:06:13,981::__init__::232::IOProcess::(_processLogs) (92002) Queuing response >Thread-18::DEBUG::2014-10-03 07:06:13,981::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 07:06:13,981::__init__::232::IOProcess::(_processLogs) (92062) Got request for method 'glob' >Thread-18::DEBUG::2014-10-03 07:06:22,554::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 07:06:22,554::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107998::WARNING::2014-10-03 07:07:22,554::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 07:07:22,554::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 07:07:22,554::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 07:07:32,555::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 07:07:32,556::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 07:07:32,556::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-107999::WARNING::2014-10-03 07:08:32,556::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 07:08:32,556::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 07:08:32,556::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 07:08:42,558::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 07:08:42,558::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108000::WARNING::2014-10-03 07:09:42,558::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 07:09:42,558::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 07:09:42,558::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 07:09:52,560::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 07:09:52,560::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108001::WARNING::2014-10-03 07:10:52,560::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 07:10:52,560::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 07:10:52,560::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 07:11:02,562::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 07:11:02,562::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108002::WARNING::2014-10-03 07:12:02,562::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 07:12:02,562::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 07:12:02,562::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 07:12:12,564::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 07:12:12,564::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108003::WARNING::2014-10-03 07:13:12,564::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 07:13:12,564::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 07:13:12,564::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 07:13:22,565::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 07:13:22,566::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 07:13:22,566::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108004::WARNING::2014-10-03 07:14:22,566::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 07:14:22,566::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 07:14:22,567::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 07:14:32,568::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 07:14:32,568::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108005::WARNING::2014-10-03 07:15:32,568::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 07:15:32,569::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 07:15:32,569::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 07:15:42,570::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 07:15:42,570::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-03 07:16:18,140::__init__::232::IOProcess::(_processLogs) (92003) Queuing response >Thread-18::DEBUG::2014-10-03 07:16:18,141::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 07:16:18,141::__init__::232::IOProcess::(_processLogs) (92063) Got request for method 'glob' >Thread-18::DEBUG::2014-10-03 07:16:18,141::__init__::232::IOProcess::(_processLogs) (92004) Queuing response >Thread-18::DEBUG::2014-10-03 07:16:18,141::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 07:16:18,141::__init__::232::IOProcess::(_processLogs) (92064) Got request for method 'glob' >Thread-108006::WARNING::2014-10-03 07:16:42,570::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 07:16:42,570::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 07:16:42,571::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 07:16:52,572::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 07:16:52,572::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108007::WARNING::2014-10-03 07:17:52,572::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 07:17:52,573::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 07:17:52,573::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 07:18:02,574::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 07:18:02,574::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108008::WARNING::2014-10-03 07:19:02,574::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 07:19:02,574::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 07:19:02,575::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 07:19:12,575::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 07:19:12,576::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 07:19:12,576::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108009::WARNING::2014-10-03 07:20:12,576::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 07:20:12,576::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 07:20:12,576::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 07:20:22,578::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 07:20:22,578::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108010::WARNING::2014-10-03 07:21:22,578::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 07:21:22,578::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 07:21:22,578::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 07:21:32,580::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 07:21:32,580::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108011::WARNING::2014-10-03 07:22:32,580::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 07:22:32,580::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 07:22:32,580::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 07:22:42,582::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 07:22:42,582::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108012::WARNING::2014-10-03 07:23:42,581::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 07:23:42,582::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 07:23:42,582::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 07:23:52,583::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 07:23:52,584::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108013::WARNING::2014-10-03 07:24:52,583::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 07:24:52,584::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 07:24:52,584::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 07:25:02,584::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 07:25:02,585::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 07:25:02,586::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108014::WARNING::2014-10-03 07:26:02,585::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 07:26:02,586::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 07:26:02,586::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 07:26:12,587::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 07:26:12,587::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-03 07:26:22,300::__init__::232::IOProcess::(_processLogs) (92005) Queuing response >Thread-18::DEBUG::2014-10-03 07:26:22,301::__init__::232::IOProcess::(_processLogs) (92006) Queuing response >Thread-18::DEBUG::2014-10-03 07:26:22,301::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 07:26:22,301::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 07:26:22,301::__init__::232::IOProcess::(_processLogs) (92065) Got request for method 'glob' >Thread-18::DEBUG::2014-10-03 07:26:22,301::__init__::232::IOProcess::(_processLogs) (92066) Got request for method 'glob' >Thread-108015::WARNING::2014-10-03 07:27:12,587::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 07:27:12,588::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 07:27:12,588::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 07:27:22,589::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 07:27:22,589::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108016::WARNING::2014-10-03 07:28:22,589::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 07:28:22,589::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 07:28:22,590::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 07:28:32,591::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 07:28:32,591::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108017::WARNING::2014-10-03 07:29:32,591::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 07:29:32,591::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 07:29:32,591::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 07:29:42,593::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 07:29:42,593::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108018::WARNING::2014-10-03 07:30:42,593::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 07:30:42,593::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 07:30:42,593::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 07:30:52,594::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 07:30:52,595::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 07:30:52,595::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108019::WARNING::2014-10-03 07:31:52,595::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 07:31:52,595::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 07:31:52,595::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 07:32:02,597::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 07:32:02,597::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108020::WARNING::2014-10-03 07:33:02,596::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 07:33:02,597::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 07:33:02,597::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 07:33:12,598::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 07:33:12,599::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108021::WARNING::2014-10-03 07:34:12,598::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 07:34:12,599::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 07:34:12,599::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 07:34:22,600::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 07:34:22,600::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108022::WARNING::2014-10-03 07:35:22,600::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 07:35:22,601::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 07:35:22,601::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 07:35:32,602::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 07:35:32,602::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-03 07:36:26,461::__init__::232::IOProcess::(_processLogs) (92008) Queuing response >Thread-18::DEBUG::2014-10-03 07:36:26,461::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 07:36:26,461::__init__::232::IOProcess::(_processLogs) (92067) Got request for method 'glob' >Thread-108023::WARNING::2014-10-03 07:36:32,602::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 07:36:32,603::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 07:36:32,603::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 07:36:42,603::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 07:36:42,604::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 07:36:42,604::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108024::WARNING::2014-10-03 07:37:42,604::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 07:37:42,605::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 07:37:42,605::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 07:37:52,606::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 07:37:52,606::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108025::WARNING::2014-10-03 07:38:52,606::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 07:38:52,607::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 07:38:52,607::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 07:39:02,608::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 07:39:02,609::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108026::WARNING::2014-10-03 07:40:02,608::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 07:40:02,609::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 07:40:02,609::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 07:40:12,610::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 07:40:12,611::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108027::WARNING::2014-10-03 07:41:12,610::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 07:41:12,611::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 07:41:12,611::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 07:41:22,612::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 07:41:22,612::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108028::WARNING::2014-10-03 07:42:22,612::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 07:42:22,613::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 07:42:22,613::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 07:42:32,613::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 07:42:32,614::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 07:42:32,614::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108029::WARNING::2014-10-03 07:43:32,614::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 07:43:32,615::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 07:43:32,615::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 07:43:42,616::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 07:43:42,616::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108030::WARNING::2014-10-03 07:44:42,616::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 07:44:42,617::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 07:44:42,617::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 07:44:52,618::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 07:44:52,619::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108031::WARNING::2014-10-03 07:45:52,618::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 07:45:52,619::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 07:45:52,619::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 07:46:02,620::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 07:46:02,621::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-03 07:46:30,620::__init__::232::IOProcess::(_processLogs) (92007) Queuing response >Thread-18::DEBUG::2014-10-03 07:46:30,621::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 07:46:30,621::__init__::232::IOProcess::(_processLogs) (92068) Got request for method 'glob' >Thread-18::DEBUG::2014-10-03 07:46:30,621::__init__::232::IOProcess::(_processLogs) (92009) Queuing response >Thread-18::DEBUG::2014-10-03 07:46:30,621::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 07:46:30,621::__init__::232::IOProcess::(_processLogs) (92069) Got request for method 'glob' >Thread-108032::WARNING::2014-10-03 07:47:02,620::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 07:47:02,621::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 07:47:02,621::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 07:47:12,623::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 07:47:12,623::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108033::WARNING::2014-10-03 07:48:12,622::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 07:48:12,623::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 07:48:12,623::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 07:48:22,623::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 07:48:22,625::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 07:48:22,625::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108034::WARNING::2014-10-03 07:49:22,625::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 07:49:22,625::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 07:49:22,625::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 07:49:32,627::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 07:49:32,627::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108035::WARNING::2014-10-03 07:50:32,626::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 07:50:32,627::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 07:50:32,627::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 07:50:42,628::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 07:50:42,629::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108036::WARNING::2014-10-03 07:51:42,628::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 07:51:42,629::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 07:51:42,629::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 07:51:52,630::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 07:51:52,630::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108037::WARNING::2014-10-03 07:52:52,630::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 07:52:52,631::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 07:52:52,631::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 07:53:02,632::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 07:53:02,632::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108038::WARNING::2014-10-03 07:54:02,632::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 07:54:02,633::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 07:54:02,633::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 07:54:12,633::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 07:54:12,634::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 07:54:12,635::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108039::WARNING::2014-10-03 07:55:12,634::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 07:55:12,635::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 07:55:12,635::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 07:55:22,636::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 07:55:22,636::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108040::WARNING::2014-10-03 07:56:22,636::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 07:56:22,637::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 07:56:22,637::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 07:56:32,638::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 07:56:32,638::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-03 07:56:36,828::__init__::232::IOProcess::(_processLogs) (92010) Queuing response >Thread-18::DEBUG::2014-10-03 07:56:36,829::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 07:56:36,829::__init__::232::IOProcess::(_processLogs) (92070) Got request for method 'glob' >Thread-18::DEBUG::2014-10-03 07:56:36,829::__init__::232::IOProcess::(_processLogs) (92011) Queuing response >Thread-18::DEBUG::2014-10-03 07:56:36,829::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 07:56:36,829::__init__::232::IOProcess::(_processLogs) (92071) Got request for method 'glob' >Thread-108041::WARNING::2014-10-03 07:57:32,638::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 07:57:32,639::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 07:57:32,639::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 07:57:42,640::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 07:57:42,640::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108042::WARNING::2014-10-03 07:58:42,640::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 07:58:42,641::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 07:58:42,641::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 07:58:52,642::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 07:58:52,642::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108043::WARNING::2014-10-03 07:59:52,642::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 07:59:52,642::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 07:59:52,643::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 08:00:02,643::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 08:00:02,644::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 08:00:02,644::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108044::WARNING::2014-10-03 08:01:02,644::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 08:01:02,645::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 08:01:02,645::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 08:01:12,646::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 08:01:12,646::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108045::WARNING::2014-10-03 08:02:12,646::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 08:02:12,647::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 08:02:12,647::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 08:02:22,648::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 08:02:22,648::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108046::WARNING::2014-10-03 08:03:22,648::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 08:03:22,648::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 08:03:22,649::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 08:03:32,650::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 08:03:32,650::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108047::WARNING::2014-10-03 08:04:32,650::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 08:04:32,650::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 08:04:32,650::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 08:04:42,652::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 08:04:42,652::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108048::WARNING::2014-10-03 08:05:42,652::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 08:05:42,652::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 08:05:42,652::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 08:05:52,653::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 08:05:52,654::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 08:05:52,654::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-03 08:06:40,989::__init__::232::IOProcess::(_processLogs) (92012) Queuing response >Thread-18::DEBUG::2014-10-03 08:06:40,989::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 08:06:40,989::__init__::232::IOProcess::(_processLogs) (92072) Got request for method 'glob' >Thread-18::DEBUG::2014-10-03 08:06:40,989::__init__::232::IOProcess::(_processLogs) (92013) Queuing response >Thread-18::DEBUG::2014-10-03 08:06:40,989::__init__::232::IOProcess::(_processLogs) (92014) Queuing response >Thread-18::DEBUG::2014-10-03 08:06:40,989::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 08:06:40,989::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 08:06:40,990::__init__::232::IOProcess::(_processLogs) (92073) Got request for method 'glob' >Thread-18::DEBUG::2014-10-03 08:06:40,990::__init__::232::IOProcess::(_processLogs) (92074) Got request for method 'glob' >Thread-108049::WARNING::2014-10-03 08:06:52,653::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 08:06:52,654::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 08:06:52,654::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 08:07:02,655::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 08:07:02,656::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108050::WARNING::2014-10-03 08:08:02,655::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 08:08:02,656::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 08:08:02,656::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 08:08:12,657::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 08:08:12,657::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108051::WARNING::2014-10-03 08:09:12,657::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 08:09:12,658::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 08:09:12,658::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 08:09:22,659::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 08:09:22,659::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108052::WARNING::2014-10-03 08:10:22,659::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 08:10:22,659::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 08:10:22,660::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 08:10:32,661::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 08:10:32,661::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108053::WARNING::2014-10-03 08:11:32,661::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 08:11:32,661::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 08:11:32,661::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 08:11:42,662::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 08:11:42,663::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 08:11:42,663::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108054::WARNING::2014-10-03 08:12:42,663::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 08:12:42,663::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 08:12:42,664::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 08:12:52,665::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 08:12:52,665::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108055::WARNING::2014-10-03 08:13:52,665::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 08:13:52,665::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 08:13:52,665::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 08:14:02,667::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 08:14:02,667::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108056::WARNING::2014-10-03 08:15:02,667::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 08:15:02,667::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 08:15:02,667::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 08:15:12,668::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 08:15:12,669::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108057::WARNING::2014-10-03 08:16:12,669::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 08:16:12,670::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 08:16:12,671::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 08:16:22,673::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 08:16:22,673::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-03 08:16:45,148::__init__::232::IOProcess::(_processLogs) (92015) Queuing response >Thread-18::DEBUG::2014-10-03 08:16:45,149::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 08:16:45,149::__init__::232::IOProcess::(_processLogs) (92075) Got request for method 'glob' >Thread-18::DEBUG::2014-10-03 08:16:45,149::__init__::232::IOProcess::(_processLogs) (92016) Queuing response >Thread-18::DEBUG::2014-10-03 08:16:45,149::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 08:16:45,149::__init__::232::IOProcess::(_processLogs) (92076) Got request for method 'glob' >Thread-108058::WARNING::2014-10-03 08:17:22,673::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 08:17:22,674::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 08:17:22,674::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 08:17:32,674::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 08:17:32,675::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 08:17:32,675::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108059::WARNING::2014-10-03 08:18:32,675::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 08:18:32,676::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 08:18:32,676::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 08:18:42,677::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 08:18:42,677::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108060::WARNING::2014-10-03 08:19:42,677::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 08:19:42,678::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 08:19:42,678::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 08:19:52,679::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 08:19:52,679::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108061::WARNING::2014-10-03 08:20:52,679::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 08:20:52,679::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 08:20:52,680::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 08:21:02,681::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 08:21:02,681::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108062::WARNING::2014-10-03 08:22:02,681::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 08:22:02,681::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 08:22:02,682::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 08:22:12,683::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 08:22:12,683::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108063::WARNING::2014-10-03 08:23:12,683::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 08:23:12,683::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 08:23:12,683::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 08:23:22,684::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 08:23:22,685::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 08:23:22,685::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108064::WARNING::2014-10-03 08:24:22,685::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 08:24:22,685::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 08:24:22,685::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 08:24:32,687::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 08:24:32,687::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108065::WARNING::2014-10-03 08:25:32,687::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 08:25:32,687::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 08:25:32,687::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 08:25:42,688::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 08:25:42,689::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108066::WARNING::2014-10-03 08:26:42,688::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 08:26:42,689::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 08:26:42,689::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 08:26:49,308::__init__::232::IOProcess::(_processLogs) (92017) Queuing response >Thread-18::DEBUG::2014-10-03 08:26:49,309::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 08:26:49,309::__init__::232::IOProcess::(_processLogs) (92077) Got request for method 'glob' >Thread-18::DEBUG::2014-10-03 08:26:49,309::__init__::232::IOProcess::(_processLogs) (92018) Queuing response >Thread-18::DEBUG::2014-10-03 08:26:49,309::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 08:26:49,309::__init__::232::IOProcess::(_processLogs) (92078) Got request for method 'glob' >Thread-18::DEBUG::2014-10-03 08:26:52,690::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 08:26:52,690::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108067::WARNING::2014-10-03 08:27:52,690::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 08:27:52,691::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 08:27:52,691::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 08:28:02,692::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 08:28:02,692::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108068::WARNING::2014-10-03 08:29:02,692::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 08:29:02,693::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 08:29:02,693::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 08:29:12,693::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 08:29:12,694::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 08:29:12,694::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108069::WARNING::2014-10-03 08:30:12,694::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 08:30:12,695::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 08:30:12,695::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 08:30:22,696::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 08:30:22,696::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108070::WARNING::2014-10-03 08:31:22,696::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 08:31:22,697::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 08:31:22,697::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 08:31:32,698::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 08:31:32,698::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108071::WARNING::2014-10-03 08:32:32,698::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 08:32:32,699::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 08:32:32,699::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 08:32:42,700::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 08:32:42,700::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108072::WARNING::2014-10-03 08:33:42,700::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 08:33:42,700::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 08:33:42,701::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 08:33:52,702::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 08:33:52,702::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108073::WARNING::2014-10-03 08:34:52,702::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 08:34:52,702::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 08:34:52,702::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 08:35:02,703::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 08:35:02,704::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 08:35:02,704::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108074::WARNING::2014-10-03 08:36:02,704::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 08:36:02,704::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 08:36:02,705::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 08:36:12,706::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 08:36:12,706::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-03 08:36:53,468::__init__::232::IOProcess::(_processLogs) (92019) Queuing response >Thread-18::DEBUG::2014-10-03 08:36:53,469::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 08:36:53,469::__init__::232::IOProcess::(_processLogs) (92079) Got request for method 'glob' >Thread-18::DEBUG::2014-10-03 08:36:53,469::__init__::232::IOProcess::(_processLogs) (92020) Queuing response >Thread-18::DEBUG::2014-10-03 08:36:53,469::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 08:36:53,469::__init__::232::IOProcess::(_processLogs) (92080) Got request for method 'glob' >Thread-108075::WARNING::2014-10-03 08:37:12,706::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 08:37:12,707::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 08:37:12,707::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 08:37:22,708::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 08:37:22,708::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108076::WARNING::2014-10-03 08:38:22,708::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 08:38:22,709::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 08:38:22,709::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 08:38:32,711::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 08:38:32,711::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108077::WARNING::2014-10-03 08:39:32,710::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 08:39:32,711::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 08:39:32,711::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 08:39:42,713::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 08:39:42,713::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108078::WARNING::2014-10-03 08:40:42,713::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 08:40:42,713::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 08:40:42,713::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 08:40:52,714::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 08:40:52,715::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 08:40:52,715::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108079::WARNING::2014-10-03 08:41:52,715::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 08:41:52,715::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 08:41:52,715::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 08:42:02,717::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 08:42:02,717::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108080::WARNING::2014-10-03 08:43:02,717::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 08:43:02,717::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 08:43:02,717::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 08:43:12,719::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 08:43:12,719::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108081::WARNING::2014-10-03 08:44:12,719::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 08:44:12,719::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 08:44:12,720::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 08:44:22,721::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 08:44:22,721::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108082::WARNING::2014-10-03 08:45:22,721::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 08:45:22,721::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 08:45:22,721::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 08:45:32,723::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 08:45:32,723::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108083::WARNING::2014-10-03 08:46:32,723::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 08:46:32,723::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 08:46:32,723::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 08:46:42,724::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 08:46:42,725::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 08:46:42,725::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-03 08:46:57,628::__init__::232::IOProcess::(_processLogs) (92021) Queuing response >Thread-18::DEBUG::2014-10-03 08:46:57,629::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 08:46:57,629::__init__::232::IOProcess::(_processLogs) (92081) Got request for method 'glob' >Thread-18::DEBUG::2014-10-03 08:46:57,629::__init__::232::IOProcess::(_processLogs) (92022) Queuing response >Thread-18::DEBUG::2014-10-03 08:46:57,629::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 08:46:57,629::__init__::232::IOProcess::(_processLogs) (92082) Got request for method 'glob' >Thread-108084::WARNING::2014-10-03 08:47:42,725::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 08:47:42,725::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 08:47:42,726::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 08:47:52,727::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 08:47:52,727::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108085::WARNING::2014-10-03 08:48:52,727::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 08:48:52,727::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 08:48:52,728::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 08:49:02,729::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 08:49:02,729::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108086::WARNING::2014-10-03 08:50:02,729::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 08:50:02,729::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 08:50:02,730::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 08:50:12,731::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 08:50:12,731::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108087::WARNING::2014-10-03 08:51:12,731::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 08:51:12,731::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 08:51:12,731::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 08:51:22,732::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 08:51:22,733::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108088::WARNING::2014-10-03 08:52:22,732::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 08:52:22,733::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 08:52:22,733::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 08:52:32,733::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 08:52:32,734::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 08:52:32,734::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108089::WARNING::2014-10-03 08:53:32,734::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 08:53:32,735::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 08:53:32,735::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 08:53:42,736::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 08:53:42,736::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108090::WARNING::2014-10-03 08:54:42,736::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 08:54:42,737::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 08:54:42,737::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 08:54:52,738::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 08:54:52,738::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108091::WARNING::2014-10-03 08:55:52,738::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 08:55:52,738::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 08:55:52,738::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 08:56:02,740::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 08:56:02,740::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-03 08:57:01,788::__init__::232::IOProcess::(_processLogs) (92023) Queuing response >Thread-18::DEBUG::2014-10-03 08:57:01,789::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 08:57:01,789::__init__::232::IOProcess::(_processLogs) (92083) Got request for method 'glob' >Thread-108092::WARNING::2014-10-03 08:57:02,740::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 08:57:02,740::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 08:57:02,740::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 08:57:12,741::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 08:57:12,741::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108093::WARNING::2014-10-03 08:58:12,741::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 08:58:12,742::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 08:58:12,742::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 08:58:22,742::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 08:58:22,743::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 08:58:22,743::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108094::WARNING::2014-10-03 08:59:22,743::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 08:59:22,744::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 08:59:22,744::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 08:59:32,745::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 08:59:32,745::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108095::WARNING::2014-10-03 09:00:32,745::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 09:00:32,745::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 09:00:32,746::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 09:00:42,747::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 09:00:42,747::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108096::WARNING::2014-10-03 09:01:42,747::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 09:01:42,747::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 09:01:42,748::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 09:01:52,749::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 09:01:52,749::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108097::WARNING::2014-10-03 09:02:52,749::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 09:02:52,749::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 09:02:52,749::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 09:03:02,750::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 09:03:02,751::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108098::WARNING::2014-10-03 09:04:02,750::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 09:04:02,751::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 09:04:02,751::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 09:04:12,751::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 09:04:12,752::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 09:04:12,753::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108099::WARNING::2014-10-03 09:05:12,752::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 09:05:12,753::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 09:05:12,753::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 09:05:22,754::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 09:05:22,754::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108100::WARNING::2014-10-03 09:06:22,754::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 09:06:22,755::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 09:06:22,755::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 09:06:32,756::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 09:06:32,756::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108101::WARNING::2014-10-03 09:07:32,756::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 09:07:32,756::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 09:07:32,757::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 09:07:42,758::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 09:07:42,758::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108102::WARNING::2014-10-03 09:08:42,758::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 09:08:42,758::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 09:08:42,758::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 09:08:52,760::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 09:08:52,760::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108103::WARNING::2014-10-03 09:09:52,760::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 09:09:52,760::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 09:09:52,760::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 09:10:02,761::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 09:10:02,761::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 09:10:02,762::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108104::WARNING::2014-10-03 09:11:02,761::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 09:11:02,762::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 09:11:02,763::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 09:11:12,765::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 09:11:12,766::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108105::WARNING::2014-10-03 09:12:12,765::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 09:12:12,766::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 09:12:12,766::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 09:12:22,767::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 09:12:22,767::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108106::WARNING::2014-10-03 09:13:22,767::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 09:13:22,768::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 09:13:22,768::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 09:13:32,769::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 09:13:32,769::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108107::WARNING::2014-10-03 09:14:32,769::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 09:14:32,770::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 09:14:32,770::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 09:14:42,771::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 09:14:42,771::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108108::WARNING::2014-10-03 09:15:42,771::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 09:15:42,772::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 09:15:42,772::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 09:15:52,772::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 09:15:52,773::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 09:15:52,773::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108109::WARNING::2014-10-03 09:16:52,773::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 09:16:52,774::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 09:16:52,774::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 09:17:02,775::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 09:17:02,776::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108110::WARNING::2014-10-03 09:18:02,775::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 09:18:02,776::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 09:18:02,776::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 09:18:12,777::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 09:18:12,777::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108111::WARNING::2014-10-03 09:19:12,777::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 09:19:12,778::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 09:19:12,778::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 09:19:22,779::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 09:19:22,779::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108112::WARNING::2014-10-03 09:20:22,779::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 09:20:22,780::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 09:20:22,780::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 09:20:32,781::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 09:20:32,781::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108113::WARNING::2014-10-03 09:21:32,781::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 09:21:32,781::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 09:21:32,782::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 09:21:42,782::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 09:21:42,783::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 09:21:42,783::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108114::WARNING::2014-10-03 09:22:42,783::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 09:22:42,783::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 09:22:42,784::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 09:22:52,785::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 09:22:52,785::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108115::WARNING::2014-10-03 09:23:52,785::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 09:23:52,785::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 09:23:52,786::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 09:24:02,787::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 09:24:02,787::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108116::WARNING::2014-10-03 09:25:02,787::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 09:25:02,787::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 09:25:02,788::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 09:25:12,789::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 09:25:12,789::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108117::WARNING::2014-10-03 09:26:12,789::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 09:26:12,789::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 09:26:12,789::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 09:26:22,791::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 09:26:22,791::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108118::WARNING::2014-10-03 09:27:22,791::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 09:27:22,791::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 09:27:22,791::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 09:27:32,792::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 09:27:32,793::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 09:27:32,793::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108119::WARNING::2014-10-03 09:28:32,793::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 09:28:32,793::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 09:28:32,793::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 09:28:42,794::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 09:28:42,795::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108120::WARNING::2014-10-03 09:29:42,794::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 09:29:42,795::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 09:29:42,795::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 09:29:52,796::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 09:29:52,797::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108121::WARNING::2014-10-03 09:30:52,796::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 09:30:52,797::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 09:30:52,797::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 09:31:02,798::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 09:31:02,798::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108122::WARNING::2014-10-03 09:32:02,798::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 09:32:02,799::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 09:32:02,799::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 09:32:12,800::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 09:32:12,800::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108123::WARNING::2014-10-03 09:33:12,800::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 09:33:12,801::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 09:33:12,801::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 09:33:22,801::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 09:33:22,802::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 09:33:22,802::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108124::WARNING::2014-10-03 09:34:22,802::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 09:34:22,803::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 09:34:22,803::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 09:34:32,804::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 09:34:32,804::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108125::WARNING::2014-10-03 09:35:32,804::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 09:35:32,805::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 09:35:32,805::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 09:35:42,806::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 09:35:42,806::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108126::WARNING::2014-10-03 09:36:42,806::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 09:36:42,807::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 09:36:42,807::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 09:36:52,808::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 09:36:52,808::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108127::WARNING::2014-10-03 09:37:52,808::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 09:37:52,808::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 09:37:52,809::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 09:38:02,810::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 09:38:02,810::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108128::WARNING::2014-10-03 09:39:02,810::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 09:39:02,810::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 09:39:02,811::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 09:39:12,811::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 09:39:12,812::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 09:39:12,812::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108129::WARNING::2014-10-03 09:40:12,812::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 09:40:12,812::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 09:40:12,813::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 09:40:22,814::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 09:40:22,814::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108130::WARNING::2014-10-03 09:41:22,814::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 09:41:22,814::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 09:41:22,814::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 09:41:32,816::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 09:41:32,816::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108131::WARNING::2014-10-03 09:42:32,816::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 09:42:32,816::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 09:42:32,816::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 09:42:42,818::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 09:42:42,818::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108132::WARNING::2014-10-03 09:43:42,818::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 09:43:42,818::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 09:43:42,818::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 09:43:52,820::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 09:43:52,820::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108133::WARNING::2014-10-03 09:44:52,820::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 09:44:52,820::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 09:44:52,820::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 09:45:02,821::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 09:45:02,822::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 09:45:02,822::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108134::WARNING::2014-10-03 09:46:02,822::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 09:46:02,822::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 09:46:02,823::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 09:46:12,824::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 09:46:12,824::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108135::WARNING::2014-10-03 09:47:12,824::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 09:47:12,824::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 09:47:12,824::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 09:47:22,826::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 09:47:22,826::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108136::WARNING::2014-10-03 09:48:22,826::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 09:48:22,826::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 09:48:22,826::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 09:48:32,828::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 09:48:32,828::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108137::WARNING::2014-10-03 09:49:32,828::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 09:49:32,828::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 09:49:32,828::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 09:49:42,830::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 09:49:42,830::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108138::WARNING::2014-10-03 09:50:42,830::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 09:50:42,830::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 09:50:42,830::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 09:50:52,831::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 09:50:52,832::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 09:50:52,832::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108139::WARNING::2014-10-03 09:51:52,831::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 09:51:52,832::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 09:51:52,832::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 09:52:02,833::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 09:52:02,833::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108140::WARNING::2014-10-03 09:53:02,833::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 09:53:02,834::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 09:53:02,834::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 09:53:12,835::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 09:53:12,835::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108141::WARNING::2014-10-03 09:54:12,835::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 09:54:12,835::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 09:54:12,836::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 09:54:22,837::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 09:54:22,837::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108142::WARNING::2014-10-03 09:55:22,837::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 09:55:22,837::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 09:55:22,837::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 09:55:32,838::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 09:55:32,838::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108143::WARNING::2014-10-03 09:56:32,838::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 09:56:32,839::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 09:56:32,839::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 09:56:42,839::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 09:56:42,840::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 09:56:42,841::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108144::WARNING::2014-10-03 09:57:42,840::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 09:57:42,841::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 09:57:42,841::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 09:57:52,842::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 09:57:52,842::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108145::WARNING::2014-10-03 09:58:52,842::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 09:58:52,843::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 09:58:52,843::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 09:59:02,844::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 09:59:02,844::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108146::WARNING::2014-10-03 10:00:02,844::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 10:00:02,845::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 10:00:02,845::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 10:00:12,846::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 10:00:12,846::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108147::WARNING::2014-10-03 10:01:12,846::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 10:01:12,846::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 10:01:12,847::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 10:01:22,848::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 10:01:22,848::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108148::WARNING::2014-10-03 10:02:22,848::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 10:02:22,848::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 10:02:22,848::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 10:02:32,849::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 10:02:32,850::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 10:02:32,850::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108149::WARNING::2014-10-03 10:03:32,849::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 10:03:32,850::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 10:03:32,850::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 10:03:42,851::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 10:03:42,851::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108150::WARNING::2014-10-03 10:04:42,851::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 10:04:42,852::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 10:04:42,852::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 10:04:52,853::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 10:04:52,853::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108151::WARNING::2014-10-03 10:05:52,853::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 10:05:52,853::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 10:05:52,854::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 10:06:02,855::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 10:06:02,855::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108152::WARNING::2014-10-03 10:07:02,855::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 10:07:02,855::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 10:07:02,855::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 10:07:12,857::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 10:07:12,857::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108153::WARNING::2014-10-03 10:08:12,857::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 10:08:12,857::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 10:08:12,857::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 10:08:22,858::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 10:08:22,859::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 10:08:22,859::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108154::WARNING::2014-10-03 10:09:22,859::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 10:09:22,859::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 10:09:22,859::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 10:09:32,861::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 10:09:32,861::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108155::WARNING::2014-10-03 10:10:32,861::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 10:10:32,861::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 10:10:32,861::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 10:10:42,862::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 10:10:42,863::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108156::WARNING::2014-10-03 10:11:42,862::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 10:11:42,863::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 10:11:42,863::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 10:11:52,864::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 10:11:52,864::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108157::WARNING::2014-10-03 10:12:52,864::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 10:12:52,865::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 10:12:52,865::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 10:13:02,866::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 10:13:02,866::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108158::WARNING::2014-10-03 10:14:02,866::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 10:14:02,867::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 10:14:02,867::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 10:14:12,867::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 10:14:12,868::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 10:14:12,868::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108159::WARNING::2014-10-03 10:15:12,868::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 10:15:12,869::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 10:15:12,869::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 10:15:22,870::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 10:15:22,871::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108160::WARNING::2014-10-03 10:16:22,870::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 10:16:22,871::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 10:16:22,871::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 10:16:32,872::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 10:16:32,873::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108161::WARNING::2014-10-03 10:17:32,872::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 10:17:32,873::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 10:17:32,873::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 10:17:35,068::__init__::232::IOProcess::(_processLogs) (92024) Queuing response >Thread-18::DEBUG::2014-10-03 10:17:35,069::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 10:17:35,069::__init__::232::IOProcess::(_processLogs) (92084) Got request for method 'glob' >Thread-18::DEBUG::2014-10-03 10:17:42,875::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 10:17:42,875::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108162::WARNING::2014-10-03 10:18:42,875::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 10:18:42,876::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 10:18:42,878::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 10:18:52,880::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 10:18:52,880::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108163::WARNING::2014-10-03 10:19:52,880::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 10:19:52,880::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 10:19:52,880::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 10:20:02,881::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 10:20:02,882::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 10:20:02,882::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108164::WARNING::2014-10-03 10:21:02,882::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 10:21:02,882::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 10:21:02,882::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 10:21:12,884::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 10:21:12,884::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108165::WARNING::2014-10-03 10:22:12,883::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 10:22:12,884::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 10:22:12,884::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 10:22:22,885::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 10:22:22,886::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108166::WARNING::2014-10-03 10:23:22,885::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 10:23:22,886::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 10:23:22,886::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 10:23:32,887::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 10:23:32,888::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108167::WARNING::2014-10-03 10:24:32,887::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 10:24:32,888::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 10:24:32,888::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 10:24:42,889::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 10:24:42,889::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108168::WARNING::2014-10-03 10:25:42,889::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 10:25:42,889::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 10:25:42,890::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 10:25:52,890::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 10:25:52,891::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 10:25:52,891::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108169::WARNING::2014-10-03 10:26:52,891::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 10:26:52,892::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 10:26:52,892::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 10:27:02,893::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 10:27:02,893::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108170::WARNING::2014-10-03 10:28:02,893::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 10:28:02,893::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 10:28:02,894::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 10:28:12,895::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 10:28:12,895::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108171::WARNING::2014-10-03 10:29:12,895::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 10:29:12,895::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 10:29:12,896::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 10:29:22,897::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 10:29:22,897::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108172::WARNING::2014-10-03 10:30:22,897::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 10:30:22,897::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 10:30:22,898::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 10:30:32,899::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 10:30:32,899::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108173::WARNING::2014-10-03 10:31:32,899::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 10:31:32,899::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 10:31:32,899::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 10:31:42,900::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 10:31:42,901::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 10:31:42,901::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108174::WARNING::2014-10-03 10:32:42,901::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 10:32:42,901::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 10:32:42,902::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 10:32:52,903::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 10:32:52,903::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108175::WARNING::2014-10-03 10:33:52,903::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 10:33:52,903::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 10:33:52,903::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 10:34:02,905::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 10:34:02,905::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108176::WARNING::2014-10-03 10:35:02,904::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 10:35:02,905::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 10:35:02,905::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 10:35:12,906::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 10:35:12,907::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108177::WARNING::2014-10-03 10:36:12,906::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 10:36:12,907::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 10:36:12,907::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 10:36:22,908::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 10:36:22,908::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108178::WARNING::2014-10-03 10:37:22,908::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 10:37:22,909::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 10:37:22,909::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 10:37:32,909::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 10:37:32,910::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 10:37:32,910::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108179::WARNING::2014-10-03 10:38:32,910::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 10:38:32,911::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 10:38:32,911::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 10:38:42,912::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 10:38:42,912::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108180::WARNING::2014-10-03 10:39:42,912::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 10:39:42,912::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 10:39:42,913::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 10:39:52,914::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 10:39:52,914::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108181::WARNING::2014-10-03 10:40:52,914::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 10:40:52,914::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 10:40:52,914::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 10:41:02,916::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 10:41:02,916::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108182::WARNING::2014-10-03 10:42:02,916::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 10:42:02,916::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 10:42:02,916::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 10:42:12,918::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 10:42:12,918::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108183::WARNING::2014-10-03 10:43:12,917::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 10:43:12,918::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 10:43:12,918::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 10:43:22,919::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 10:43:22,920::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 10:43:22,920::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108184::WARNING::2014-10-03 10:44:22,920::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 10:44:22,920::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 10:44:22,920::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 10:44:32,921::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 10:44:32,922::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108185::WARNING::2014-10-03 10:45:32,921::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 10:45:32,922::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 10:45:32,922::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 10:45:42,924::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 10:45:42,924::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108186::WARNING::2014-10-03 10:46:42,923::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 10:46:42,924::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 10:46:42,924::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 10:46:52,926::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 10:46:52,926::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108187::WARNING::2014-10-03 10:47:52,925::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 10:47:52,926::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 10:47:52,927::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 10:48:02,928::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 10:48:02,928::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108188::WARNING::2014-10-03 10:49:02,928::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 10:49:02,928::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 10:49:02,929::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 10:49:12,929::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 10:49:12,930::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 10:49:12,930::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108189::WARNING::2014-10-03 10:50:12,930::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 10:50:12,930::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 10:50:12,931::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 10:50:22,932::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 10:50:22,932::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108190::WARNING::2014-10-03 10:51:22,932::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 10:51:22,932::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 10:51:22,932::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 10:51:32,934::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 10:51:32,934::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108191::WARNING::2014-10-03 10:52:32,933::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 10:52:32,934::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 10:52:32,934::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 10:52:42,935::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 10:52:42,935::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108192::WARNING::2014-10-03 10:53:42,935::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 10:53:42,935::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 10:53:42,936::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 10:53:52,937::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 10:53:52,937::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108193::WARNING::2014-10-03 10:54:52,937::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 10:54:52,937::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 10:54:52,937::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 10:55:02,938::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 10:55:02,939::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 10:55:02,939::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108194::WARNING::2014-10-03 10:56:02,939::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 10:56:02,939::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 10:56:02,939::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 10:56:12,941::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 10:56:12,941::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108195::WARNING::2014-10-03 10:57:12,940::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 10:57:12,941::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 10:57:12,941::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 10:57:22,942::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 10:57:22,942::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-03 10:57:51,709::__init__::232::IOProcess::(_processLogs) (92025) Queuing response >Thread-18::DEBUG::2014-10-03 10:57:51,709::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 10:57:51,709::__init__::232::IOProcess::(_processLogs) (92085) Got request for method 'glob' >Thread-108196::WARNING::2014-10-03 10:58:22,942::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 10:58:22,943::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 10:58:22,943::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 10:58:32,944::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 10:58:32,944::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108197::WARNING::2014-10-03 10:59:32,944::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 10:59:32,944::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 10:59:32,945::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 10:59:42,946::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 10:59:42,946::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108198::WARNING::2014-10-03 11:00:42,946::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 11:00:42,946::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 11:00:42,946::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 11:00:52,947::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 11:00:52,948::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 11:00:52,948::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108199::WARNING::2014-10-03 11:01:52,948::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 11:01:52,949::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 11:01:52,950::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 11:02:02,952::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 11:02:02,952::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108200::WARNING::2014-10-03 11:03:02,952::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 11:03:02,952::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 11:03:02,953::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 11:03:12,954::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 11:03:12,954::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108201::WARNING::2014-10-03 11:04:12,954::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 11:04:12,954::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 11:04:12,954::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 11:04:22,956::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 11:04:22,956::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108202::WARNING::2014-10-03 11:05:22,955::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 11:05:22,956::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 11:05:22,956::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 11:05:32,957::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 11:05:32,958::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108203::WARNING::2014-10-03 11:06:32,957::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 11:06:32,958::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 11:06:32,958::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 11:06:42,958::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 11:06:42,959::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 11:06:42,959::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108204::WARNING::2014-10-03 11:07:42,959::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 11:07:42,960::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 11:07:42,960::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 11:07:52,961::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 11:07:52,961::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-03 11:07:55,868::__init__::232::IOProcess::(_processLogs) (92026) Queuing response >Thread-18::DEBUG::2014-10-03 11:07:55,869::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 11:07:55,869::__init__::232::IOProcess::(_processLogs) (92086) Got request for method 'glob' >Thread-108205::WARNING::2014-10-03 11:08:52,961::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 11:08:52,962::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 11:08:52,962::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 11:09:02,963::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 11:09:02,963::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108206::WARNING::2014-10-03 11:10:02,963::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 11:10:02,964::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 11:10:02,964::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 11:10:12,965::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 11:10:12,965::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108207::WARNING::2014-10-03 11:11:12,965::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 11:11:12,965::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 11:11:12,966::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 11:11:22,967::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 11:11:22,967::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108208::WARNING::2014-10-03 11:12:22,967::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 11:12:22,967::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 11:12:22,967::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 11:12:32,968::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 11:12:32,969::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 11:12:32,969::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108209::WARNING::2014-10-03 11:13:32,969::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 11:13:32,969::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 11:13:32,969::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 11:13:42,971::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 11:13:42,971::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108210::WARNING::2014-10-03 11:14:42,970::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 11:14:42,971::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 11:14:42,971::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 11:14:52,972::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 11:14:52,973::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108211::WARNING::2014-10-03 11:15:52,972::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 11:15:52,973::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 11:15:52,973::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 11:16:02,974::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 11:16:02,974::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108212::WARNING::2014-10-03 11:17:02,974::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 11:17:02,975::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 11:17:02,975::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 11:17:12,976::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 11:17:12,976::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-03 11:18:00,028::__init__::232::IOProcess::(_processLogs) (92027) Queuing response >Thread-18::DEBUG::2014-10-03 11:18:00,029::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 11:18:00,029::__init__::232::IOProcess::(_processLogs) (92087) Got request for method 'glob' >Thread-108213::WARNING::2014-10-03 11:18:12,976::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 11:18:12,977::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 11:18:12,977::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 11:18:22,977::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 11:18:22,978::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 11:18:22,979::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108214::WARNING::2014-10-03 11:19:22,978::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 11:19:22,979::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 11:19:22,979::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 11:19:32,980::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 11:19:32,981::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108215::WARNING::2014-10-03 11:20:32,980::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 11:20:32,981::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 11:20:32,981::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 11:20:42,982::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 11:20:42,982::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108216::WARNING::2014-10-03 11:21:42,982::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 11:21:42,982::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 11:21:42,983::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 11:21:52,984::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 11:21:52,984::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108217::WARNING::2014-10-03 11:22:52,984::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 11:22:52,984::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 11:22:52,985::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 11:23:02,986::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 11:23:02,986::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108218::WARNING::2014-10-03 11:24:02,986::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 11:24:02,986::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 11:24:02,987::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 11:24:12,987::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 11:24:12,988::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 11:24:12,988::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108219::WARNING::2014-10-03 11:25:12,988::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 11:25:12,988::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 11:25:12,989::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 11:25:22,990::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 11:25:22,990::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108220::WARNING::2014-10-03 11:26:22,990::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 11:26:22,990::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 11:26:22,991::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 11:26:32,992::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 11:26:32,992::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108221::WARNING::2014-10-03 11:27:32,992::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 11:27:32,992::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 11:27:32,993::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 11:27:42,994::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 11:27:42,994::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108222::WARNING::2014-10-03 11:28:42,994::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 11:28:42,994::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 11:28:42,995::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 11:28:52,996::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 11:28:52,996::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108223::WARNING::2014-10-03 11:29:52,996::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 11:29:52,996::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 11:29:52,996::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 11:30:02,997::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 11:30:02,998::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 11:30:02,998::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108224::WARNING::2014-10-03 11:31:02,998::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 11:31:02,998::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 11:31:02,998::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 11:31:12,999::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 11:31:13,000::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108225::WARNING::2014-10-03 11:32:12,999::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 11:32:13,000::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 11:32:13,000::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 11:32:23,001::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 11:32:23,001::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108226::WARNING::2014-10-03 11:33:23,001::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 11:33:23,002::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 11:33:23,002::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 11:33:33,003::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 11:33:33,003::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108227::WARNING::2014-10-03 11:34:33,003::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 11:34:33,004::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 11:34:33,004::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 11:34:43,005::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 11:34:43,005::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108228::WARNING::2014-10-03 11:35:43,005::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 11:35:43,005::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 11:35:43,006::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 11:35:53,006::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 11:35:53,007::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 11:35:53,007::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108229::WARNING::2014-10-03 11:36:53,007::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 11:36:53,007::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 11:36:53,008::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 11:37:03,009::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 11:37:03,009::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108230::WARNING::2014-10-03 11:38:03,009::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 11:38:03,009::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 11:38:03,009::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 11:38:13,011::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 11:38:13,011::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108231::WARNING::2014-10-03 11:39:13,011::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 11:39:13,011::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 11:39:13,011::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 11:39:23,012::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 11:39:23,013::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108232::WARNING::2014-10-03 11:40:23,012::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 11:40:23,013::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 11:40:23,013::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 11:40:33,014::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 11:40:33,015::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108233::WARNING::2014-10-03 11:41:33,014::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 11:41:33,015::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 11:41:33,015::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 11:41:43,015::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 11:41:43,016::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 11:41:43,017::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108234::WARNING::2014-10-03 11:42:43,016::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 11:42:43,017::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 11:42:43,017::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 11:42:53,018::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 11:42:53,018::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108235::WARNING::2014-10-03 11:43:53,018::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 11:43:53,019::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 11:43:53,019::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 11:44:03,020::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 11:44:03,020::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108236::WARNING::2014-10-03 11:45:03,020::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 11:45:03,021::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 11:45:03,021::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 11:45:13,022::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 11:45:13,022::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108237::WARNING::2014-10-03 11:46:13,022::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 11:46:13,022::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 11:46:13,023::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 11:46:23,024::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 11:46:23,024::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108238::WARNING::2014-10-03 11:47:23,024::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 11:47:23,024::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 11:47:23,024::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 11:47:33,025::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 11:47:33,026::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 11:47:33,026::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108239::WARNING::2014-10-03 11:48:33,026::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 11:48:33,026::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 11:48:33,027::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 11:48:43,028::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 11:48:43,028::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108240::WARNING::2014-10-03 11:49:43,028::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 11:49:43,028::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 11:49:43,028::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 11:49:53,030::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 11:49:53,030::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108241::WARNING::2014-10-03 11:50:53,029::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 11:50:53,030::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 11:50:53,030::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 11:51:03,031::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 11:51:03,031::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108242::WARNING::2014-10-03 11:52:03,031::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 11:52:03,032::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 11:52:03,032::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 11:52:13,033::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 11:52:13,033::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108243::WARNING::2014-10-03 11:53:13,033::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 11:53:13,033::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 11:53:13,034::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 11:53:23,034::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 11:53:23,035::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 11:53:23,035::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108244::WARNING::2014-10-03 11:54:23,035::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 11:54:23,035::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 11:54:23,036::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 11:54:33,037::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 11:54:33,037::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108245::WARNING::2014-10-03 11:55:33,037::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 11:55:33,037::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 11:55:33,037::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 11:55:43,039::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 11:55:43,039::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108246::WARNING::2014-10-03 11:56:43,038::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 11:56:43,039::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 11:56:43,039::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 11:56:53,040::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 11:56:53,040::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108247::WARNING::2014-10-03 11:57:53,040::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 11:57:53,041::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 11:57:53,041::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 11:58:03,042::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 11:58:03,042::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-03 11:58:16,668::__init__::232::IOProcess::(_processLogs) (92028) Queuing response >Thread-18::DEBUG::2014-10-03 11:58:16,669::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 11:58:16,669::__init__::232::IOProcess::(_processLogs) (92088) Got request for method 'glob' >Thread-108248::WARNING::2014-10-03 11:59:03,042::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 11:59:03,043::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 11:59:03,043::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 11:59:13,043::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 11:59:13,044::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 11:59:13,044::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108249::WARNING::2014-10-03 12:00:13,044::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 12:00:13,045::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 12:00:13,045::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 12:00:23,046::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 12:00:23,046::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108250::WARNING::2014-10-03 12:01:23,046::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 12:01:23,047::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 12:01:23,047::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 12:01:33,048::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 12:01:33,048::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108251::WARNING::2014-10-03 12:02:33,048::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 12:02:33,048::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 12:02:33,049::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 12:02:43,050::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 12:02:43,050::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108252::WARNING::2014-10-03 12:03:43,050::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 12:03:43,050::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 12:03:43,050::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 12:03:53,052::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 12:03:53,052::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108253::WARNING::2014-10-03 12:04:53,052::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 12:04:53,052::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 12:04:53,052::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 12:05:03,053::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 12:05:03,054::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 12:05:03,054::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108254::WARNING::2014-10-03 12:06:03,054::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 12:06:03,054::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 12:06:03,054::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 12:06:13,055::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 12:06:13,056::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108255::WARNING::2014-10-03 12:07:13,055::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 12:07:13,056::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 12:07:13,056::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 12:07:23,057::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 12:07:23,058::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-03 12:08:20,828::__init__::232::IOProcess::(_processLogs) (92029) Queuing response >Thread-18::DEBUG::2014-10-03 12:08:20,829::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 12:08:20,829::__init__::232::IOProcess::(_processLogs) (92089) Got request for method 'glob' >Thread-108256::WARNING::2014-10-03 12:08:23,057::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 12:08:23,058::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 12:08:23,058::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 12:08:33,059::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 12:08:33,059::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108257::WARNING::2014-10-03 12:09:33,059::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 12:09:33,060::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 12:09:33,060::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 12:09:43,061::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 12:09:43,061::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108258::WARNING::2014-10-03 12:10:43,061::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 12:10:43,062::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 12:10:43,062::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 12:10:53,062::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 12:10:53,063::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 12:10:53,063::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108259::WARNING::2014-10-03 12:11:53,063::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 12:11:53,064::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 12:11:53,064::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 12:12:03,065::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 12:12:03,065::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108260::WARNING::2014-10-03 12:13:03,065::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 12:13:03,065::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 12:13:03,066::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 12:13:13,067::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 12:13:13,067::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108261::WARNING::2014-10-03 12:14:13,067::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 12:14:13,067::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 12:14:13,068::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 12:14:23,069::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 12:14:23,069::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108262::WARNING::2014-10-03 12:15:23,069::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 12:15:23,069::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 12:15:23,069::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 12:15:33,071::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 12:15:33,071::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108263::WARNING::2014-10-03 12:16:33,071::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 12:16:33,071::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 12:16:33,071::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 12:16:43,072::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 12:16:43,073::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 12:16:43,073::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108264::WARNING::2014-10-03 12:17:43,073::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 12:17:43,073::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 12:17:43,073::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 12:17:53,075::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 12:17:53,075::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108265::WARNING::2014-10-03 12:18:53,075::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 12:18:53,075::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 12:18:53,075::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 12:19:03,077::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 12:19:03,077::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108266::WARNING::2014-10-03 12:20:03,076::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 12:20:03,077::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 12:20:03,077::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 12:20:13,078::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 12:20:13,079::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108267::WARNING::2014-10-03 12:21:13,078::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 12:21:13,079::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 12:21:13,079::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 12:21:23,080::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 12:21:23,080::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108268::WARNING::2014-10-03 12:22:23,080::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 12:22:23,081::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 12:22:23,081::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 12:22:33,081::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 12:22:33,082::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 12:22:33,082::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108269::WARNING::2014-10-03 12:23:33,082::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 12:23:33,083::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 12:23:33,083::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 12:23:43,084::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 12:23:43,084::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108270::WARNING::2014-10-03 12:24:43,084::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 12:24:43,085::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 12:24:43,085::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 12:24:53,086::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 12:24:53,086::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108271::WARNING::2014-10-03 12:25:53,086::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 12:25:53,086::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 12:25:53,087::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 12:26:03,088::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 12:26:03,088::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108272::WARNING::2014-10-03 12:27:03,088::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 12:27:03,088::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 12:27:03,088::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 12:27:13,090::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 12:27:13,090::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108273::WARNING::2014-10-03 12:28:13,090::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 12:28:13,090::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 12:28:13,090::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 12:28:23,091::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 12:28:23,092::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 12:28:23,092::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108274::WARNING::2014-10-03 12:29:23,092::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 12:29:23,092::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 12:29:23,092::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 12:29:33,094::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 12:29:33,094::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108275::WARNING::2014-10-03 12:30:33,094::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 12:30:33,094::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 12:30:33,094::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 12:30:43,096::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 12:30:43,096::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108276::WARNING::2014-10-03 12:31:43,095::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 12:31:43,096::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 12:31:43,096::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 12:31:53,097::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 12:31:53,098::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108277::WARNING::2014-10-03 12:32:53,097::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 12:32:53,098::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 12:32:53,098::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 12:33:03,099::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 12:33:03,099::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108278::WARNING::2014-10-03 12:34:03,099::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 12:34:03,100::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 12:34:03,100::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 12:34:13,100::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 12:34:13,101::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 12:34:13,101::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108279::WARNING::2014-10-03 12:35:13,101::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 12:35:13,102::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 12:35:13,102::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 12:35:23,103::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 12:35:23,103::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108280::WARNING::2014-10-03 12:36:23,103::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 12:36:23,103::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 12:36:23,104::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 12:36:33,105::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 12:36:33,105::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108281::WARNING::2014-10-03 12:37:33,105::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 12:37:33,105::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 12:37:33,106::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 12:37:43,107::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 12:37:43,107::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108282::WARNING::2014-10-03 12:38:43,107::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 12:38:43,107::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 12:38:43,107::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 12:38:53,109::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 12:38:53,109::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108283::WARNING::2014-10-03 12:39:53,109::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 12:39:53,109::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 12:39:53,110::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 12:40:03,110::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 12:40:03,111::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 12:40:03,111::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108284::WARNING::2014-10-03 12:41:03,111::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 12:41:03,111::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 12:41:03,112::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 12:41:13,113::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 12:41:13,113::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108285::WARNING::2014-10-03 12:42:13,113::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 12:42:13,113::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 12:42:13,113::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 12:42:23,115::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 12:42:23,115::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108286::WARNING::2014-10-03 12:43:23,115::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 12:43:23,115::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 12:43:23,115::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 12:43:33,117::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 12:43:33,117::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108287::WARNING::2014-10-03 12:44:33,116::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 12:44:33,117::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 12:44:33,117::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 12:44:43,118::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 12:44:43,119::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108288::WARNING::2014-10-03 12:45:43,118::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 12:45:43,119::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 12:45:43,119::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 12:45:53,120::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 12:45:53,121::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 12:45:53,121::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108289::WARNING::2014-10-03 12:46:53,120::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 12:46:53,121::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 12:46:53,121::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 12:47:03,122::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 12:47:03,123::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108290::WARNING::2014-10-03 12:48:03,122::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 12:48:03,123::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 12:48:03,123::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 12:48:13,124::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 12:48:13,125::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108291::WARNING::2014-10-03 12:49:13,124::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 12:49:13,125::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 12:49:13,125::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 12:49:23,126::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 12:49:23,127::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108292::WARNING::2014-10-03 12:50:23,126::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 12:50:23,127::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 12:50:23,127::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 12:50:33,128::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 12:50:33,128::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108293::WARNING::2014-10-03 12:51:33,128::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 12:51:33,129::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 12:51:33,129::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 12:51:43,129::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 12:51:43,130::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 12:51:43,130::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108294::WARNING::2014-10-03 12:52:43,130::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 12:52:43,131::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 12:52:43,131::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 12:52:53,132::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 12:52:53,132::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108295::WARNING::2014-10-03 12:53:53,132::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 12:53:53,133::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 12:53:53,133::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 12:54:03,134::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 12:54:03,134::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108296::WARNING::2014-10-03 12:55:03,134::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 12:55:03,134::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 12:55:03,135::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 12:55:13,136::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 12:55:13,136::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108297::WARNING::2014-10-03 12:56:13,136::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 12:56:13,136::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 12:56:13,137::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 12:56:23,138::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 12:56:23,138::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108298::WARNING::2014-10-03 12:57:23,138::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 12:57:23,138::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 12:57:23,139::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 12:57:33,139::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 12:57:33,140::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 12:57:33,140::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108299::WARNING::2014-10-03 12:58:33,140::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 12:58:33,140::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 12:58:33,140::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 12:58:43,142::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 12:58:43,142::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108300::WARNING::2014-10-03 12:59:43,142::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 12:59:43,142::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 12:59:43,142::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 12:59:53,144::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 12:59:53,144::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108301::WARNING::2014-10-03 13:00:53,144::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 13:00:53,144::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 13:00:53,144::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 13:01:03,145::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 13:01:03,146::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108302::WARNING::2014-10-03 13:02:03,145::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 13:02:03,146::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 13:02:03,146::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 13:02:13,147::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 13:02:13,147::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108303::WARNING::2014-10-03 13:03:13,147::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 13:03:13,148::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 13:03:13,148::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 13:03:23,148::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 13:03:23,149::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 13:03:23,149::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108304::WARNING::2014-10-03 13:04:23,149::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 13:04:23,150::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 13:04:23,150::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 13:04:33,151::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 13:04:33,151::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108305::WARNING::2014-10-03 13:05:33,151::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 13:05:33,152::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 13:05:33,152::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 13:05:43,153::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 13:05:43,153::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108306::WARNING::2014-10-03 13:06:43,153::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 13:06:43,154::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 13:06:43,154::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 13:06:53,155::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 13:06:53,155::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108307::WARNING::2014-10-03 13:07:53,155::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 13:07:53,155::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 13:07:53,156::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 13:08:03,157::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 13:08:03,157::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108308::WARNING::2014-10-03 13:09:03,157::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 13:09:03,157::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 13:09:03,157::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 13:09:13,158::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 13:09:13,159::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 13:09:13,159::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108309::WARNING::2014-10-03 13:10:13,159::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 13:10:13,159::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 13:10:13,160::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 13:10:23,161::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 13:10:23,161::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108310::WARNING::2014-10-03 13:11:23,161::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 13:11:23,161::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 13:11:23,161::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 13:11:33,163::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 13:11:33,163::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108311::WARNING::2014-10-03 13:12:33,162::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 13:12:33,163::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 13:12:33,163::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 13:12:43,165::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 13:12:43,165::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108312::WARNING::2014-10-03 13:13:43,164::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 13:13:43,165::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 13:13:43,165::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 13:13:53,166::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 13:13:53,167::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108313::WARNING::2014-10-03 13:14:53,166::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 13:14:53,167::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 13:14:53,167::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 13:15:03,168::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 13:15:03,169::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 13:15:03,169::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108314::WARNING::2014-10-03 13:16:03,169::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 13:16:03,169::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 13:16:03,169::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 13:16:13,171::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 13:16:13,171::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108315::WARNING::2014-10-03 13:17:13,171::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 13:17:13,171::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 13:17:13,171::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 13:17:23,173::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 13:17:23,173::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108316::WARNING::2014-10-03 13:18:23,173::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 13:18:23,173::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 13:18:23,173::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 13:18:33,175::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 13:18:33,175::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108317::WARNING::2014-10-03 13:19:33,175::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 13:19:33,175::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 13:19:33,175::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 13:19:43,177::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 13:19:43,177::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108318::WARNING::2014-10-03 13:20:43,177::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 13:20:43,177::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 13:20:43,177::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 13:20:53,178::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 13:20:53,179::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 13:20:53,179::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108319::WARNING::2014-10-03 13:21:53,179::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 13:21:53,179::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 13:21:53,179::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 13:22:03,181::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 13:22:03,181::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108320::WARNING::2014-10-03 13:23:03,180::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 13:23:03,181::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 13:23:03,181::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 13:23:13,183::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 13:23:13,183::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108321::WARNING::2014-10-03 13:24:13,182::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 13:24:13,183::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 13:24:13,183::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 13:24:23,184::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 13:24:23,184::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108322::WARNING::2014-10-03 13:25:23,184::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 13:25:23,185::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 13:25:23,185::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 13:25:33,186::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 13:25:33,186::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108323::WARNING::2014-10-03 13:26:33,186::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 13:26:33,187::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 13:26:33,187::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 13:26:43,187::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 13:26:43,188::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 13:26:43,188::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108324::WARNING::2014-10-03 13:27:43,188::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 13:27:43,189::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 13:27:43,189::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 13:27:53,190::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 13:27:53,190::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108325::WARNING::2014-10-03 13:28:53,190::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 13:28:53,191::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 13:28:53,191::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 13:29:03,192::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 13:29:03,192::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108326::WARNING::2014-10-03 13:30:03,192::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 13:30:03,192::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 13:30:03,193::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 13:30:13,194::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 13:30:13,194::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108327::WARNING::2014-10-03 13:31:13,194::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 13:31:13,194::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 13:31:13,195::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 13:31:23,196::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 13:31:23,196::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108328::WARNING::2014-10-03 13:32:23,196::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 13:32:23,196::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 13:32:23,196::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 13:32:33,197::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 13:32:33,198::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 13:32:33,198::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108329::WARNING::2014-10-03 13:33:33,198::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 13:33:33,198::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 13:33:33,198::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 13:33:43,199::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 13:33:43,200::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108330::WARNING::2014-10-03 13:34:43,199::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 13:34:43,200::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 13:34:43,200::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 13:34:53,201::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 13:34:53,201::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108331::WARNING::2014-10-03 13:35:53,201::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 13:35:53,202::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 13:35:53,202::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 13:36:03,203::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 13:36:03,203::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108332::WARNING::2014-10-03 13:37:03,203::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 13:37:03,203::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 13:37:03,204::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 13:37:13,205::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 13:37:13,205::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108333::WARNING::2014-10-03 13:38:13,205::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 13:38:13,205::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 13:38:13,205::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 13:38:23,206::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 13:38:23,207::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 13:38:23,207::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-03 13:38:58,268::__init__::232::IOProcess::(_processLogs) (92030) Queuing response >Thread-18::DEBUG::2014-10-03 13:38:58,269::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 13:38:58,269::__init__::232::IOProcess::(_processLogs) (92090) Got request for method 'glob' >Thread-108334::WARNING::2014-10-03 13:39:23,207::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 13:39:23,208::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 13:39:23,208::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 13:39:33,209::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 13:39:33,209::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108335::WARNING::2014-10-03 13:40:33,209::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 13:40:33,210::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 13:40:33,210::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 13:40:43,211::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 13:40:43,211::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108336::WARNING::2014-10-03 13:41:43,211::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 13:41:43,211::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 13:41:43,212::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 13:41:53,213::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 13:41:53,213::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108337::WARNING::2014-10-03 13:42:53,213::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 13:42:53,213::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 13:42:53,213::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 13:43:03,214::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 13:43:03,215::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108338::WARNING::2014-10-03 13:44:03,214::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 13:44:03,215::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 13:44:03,215::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 13:44:13,215::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 13:44:13,216::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 13:44:13,217::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108339::WARNING::2014-10-03 13:45:13,216::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 13:45:13,217::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 13:45:13,217::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 13:45:23,218::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 13:45:23,218::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108340::WARNING::2014-10-03 13:46:23,218::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 13:46:23,219::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 13:46:23,219::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 13:46:33,220::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 13:46:33,220::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108341::WARNING::2014-10-03 13:47:33,220::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 13:47:33,220::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 13:47:33,221::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 13:47:43,222::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 13:47:43,222::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108342::WARNING::2014-10-03 13:48:43,222::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 13:48:43,222::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 13:48:43,223::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 13:48:53,224::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 13:48:53,224::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-03 13:49:02,428::__init__::232::IOProcess::(_processLogs) (92031) Queuing response >Thread-18::DEBUG::2014-10-03 13:49:02,429::__init__::232::IOProcess::(_processLogs) (92032) Queuing response >Thread-18::DEBUG::2014-10-03 13:49:02,429::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 13:49:02,429::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 13:49:02,429::__init__::232::IOProcess::(_processLogs) (92091) Got request for method 'glob' >Thread-18::DEBUG::2014-10-03 13:49:02,429::__init__::232::IOProcess::(_processLogs) (92092) Got request for method 'glob' >Thread-108343::WARNING::2014-10-03 13:49:53,224::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 13:49:53,224::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 13:49:53,225::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 13:50:03,225::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 13:50:03,226::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 13:50:03,226::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108344::WARNING::2014-10-03 13:51:03,226::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 13:51:03,226::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 13:51:03,227::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 13:51:13,228::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 13:51:13,228::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108345::WARNING::2014-10-03 13:52:13,228::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 13:52:13,228::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 13:52:13,228::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 13:52:23,230::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 13:52:23,230::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108346::WARNING::2014-10-03 13:53:23,230::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 13:53:23,230::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 13:53:23,230::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 13:53:33,232::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 13:53:33,232::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108347::WARNING::2014-10-03 13:54:33,231::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 13:54:33,232::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 13:54:33,232::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 13:54:43,235::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 13:54:43,235::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108348::WARNING::2014-10-03 13:55:43,234::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 13:55:43,235::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 13:55:43,235::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 13:55:53,236::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 13:55:53,237::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 13:55:53,237::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108349::WARNING::2014-10-03 13:56:53,236::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 13:56:53,237::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 13:56:53,237::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 13:57:03,238::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 13:57:03,239::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108350::WARNING::2014-10-03 13:58:03,238::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 13:58:03,239::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 13:58:03,239::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 13:58:13,240::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 13:58:13,240::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-03 13:59:06,588::__init__::232::IOProcess::(_processLogs) (92033) Queuing response >Thread-18::DEBUG::2014-10-03 13:59:06,589::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 13:59:06,589::__init__::232::IOProcess::(_processLogs) (92093) Got request for method 'glob' >Thread-18::DEBUG::2014-10-03 13:59:06,589::__init__::232::IOProcess::(_processLogs) (92034) Queuing response >Thread-18::DEBUG::2014-10-03 13:59:06,589::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 13:59:06,589::__init__::232::IOProcess::(_processLogs) (92094) Got request for method 'glob' >Thread-108351::WARNING::2014-10-03 13:59:13,240::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 13:59:13,241::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 13:59:13,241::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 13:59:23,242::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 13:59:23,242::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108352::WARNING::2014-10-03 14:00:23,242::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 14:00:23,243::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 14:00:23,243::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 14:00:33,244::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 14:00:33,244::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108353::WARNING::2014-10-03 14:01:33,244::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 14:01:33,244::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 14:01:33,245::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 14:01:43,245::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 14:01:43,246::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 14:01:43,246::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108354::WARNING::2014-10-03 14:02:43,246::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 14:02:43,246::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 14:02:43,247::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 14:02:53,248::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 14:02:53,248::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108355::WARNING::2014-10-03 14:03:53,248::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 14:03:53,248::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 14:03:53,249::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 14:04:03,250::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 14:04:03,250::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108356::WARNING::2014-10-03 14:05:03,250::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 14:05:03,250::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 14:05:03,250::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 14:05:13,252::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 14:05:13,252::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108357::WARNING::2014-10-03 14:06:13,251::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 14:06:13,252::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 14:06:13,252::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 14:06:23,253::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 14:06:23,254::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108358::WARNING::2014-10-03 14:07:23,253::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 14:07:23,254::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 14:07:23,254::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 14:07:33,254::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 14:07:33,256::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 14:07:33,256::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108359::WARNING::2014-10-03 14:08:33,255::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 14:08:33,256::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 14:08:33,256::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 14:08:43,257::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 14:08:43,258::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-03 14:09:12,796::__init__::232::IOProcess::(_processLogs) (92035) Queuing response >Thread-18::DEBUG::2014-10-03 14:09:12,797::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 14:09:12,797::__init__::232::IOProcess::(_processLogs) (92095) Got request for method 'glob' >Thread-18::DEBUG::2014-10-03 14:09:12,797::__init__::232::IOProcess::(_processLogs) (92036) Queuing response >Thread-18::DEBUG::2014-10-03 14:09:12,797::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 14:09:12,797::__init__::232::IOProcess::(_processLogs) (92096) Got request for method 'glob' >Thread-108360::WARNING::2014-10-03 14:09:43,257::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 14:09:43,258::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 14:09:43,258::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 14:09:53,260::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 14:09:53,260::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108361::WARNING::2014-10-03 14:10:53,259::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 14:10:53,260::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 14:10:53,260::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 14:11:03,262::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 14:11:03,262::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108362::WARNING::2014-10-03 14:12:03,261::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 14:12:03,262::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 14:12:03,262::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 14:12:13,264::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 14:12:13,264::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108363::WARNING::2014-10-03 14:13:13,263::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 14:13:13,264::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 14:13:13,264::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 14:13:23,265::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 14:13:23,266::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 14:13:23,266::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108364::WARNING::2014-10-03 14:14:23,266::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 14:14:23,266::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 14:14:23,266::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 14:14:33,268::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 14:14:33,268::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108365::WARNING::2014-10-03 14:15:33,268::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 14:15:33,268::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 14:15:33,269::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 14:15:43,270::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 14:15:43,270::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108366::WARNING::2014-10-03 14:16:43,270::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 14:16:43,270::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 14:16:43,271::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 14:16:53,272::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 14:16:53,272::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108367::WARNING::2014-10-03 14:17:53,272::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 14:17:53,272::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 14:17:53,273::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 14:18:03,274::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 14:18:03,274::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108368::WARNING::2014-10-03 14:19:03,274::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 14:19:03,274::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 14:19:03,275::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 14:19:13,275::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 14:19:13,276::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 14:19:13,276::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-03 14:19:16,956::__init__::232::IOProcess::(_processLogs) (92037) Queuing response >Thread-18::DEBUG::2014-10-03 14:19:16,956::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 14:19:16,957::__init__::232::IOProcess::(_processLogs) (92097) Got request for method 'glob' >Thread-18::DEBUG::2014-10-03 14:19:16,957::__init__::232::IOProcess::(_processLogs) (92038) Queuing response >Thread-18::DEBUG::2014-10-03 14:19:16,957::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 14:19:16,957::__init__::232::IOProcess::(_processLogs) (92098) Got request for method 'glob' >Thread-108369::WARNING::2014-10-03 14:20:13,276::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 14:20:13,277::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 14:20:13,277::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 14:20:23,278::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 14:20:23,278::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108370::WARNING::2014-10-03 14:21:23,278::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 14:21:23,279::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 14:21:23,279::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 14:21:33,280::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 14:21:33,280::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108371::WARNING::2014-10-03 14:22:33,280::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 14:22:33,280::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 14:22:33,280::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 14:22:43,282::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 14:22:43,282::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108372::WARNING::2014-10-03 14:23:43,281::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 14:23:43,282::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 14:23:43,282::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 14:23:53,283::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 14:23:53,284::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108373::WARNING::2014-10-03 14:24:53,283::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 14:24:53,284::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 14:24:53,284::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 14:25:03,284::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 14:25:03,285::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 14:25:03,285::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108374::WARNING::2014-10-03 14:26:03,285::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 14:26:03,286::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 14:26:03,286::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 14:26:13,287::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 14:26:13,287::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108375::WARNING::2014-10-03 14:27:13,287::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 14:27:13,287::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 14:27:13,287::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 14:27:23,289::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 14:27:23,289::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108376::WARNING::2014-10-03 14:28:23,289::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 14:28:23,289::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 14:28:23,289::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 14:28:33,290::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 14:28:33,291::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-03 14:29:23,164::__init__::232::IOProcess::(_processLogs) (92039) Queuing response >Thread-18::DEBUG::2014-10-03 14:29:23,165::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 14:29:23,165::__init__::232::IOProcess::(_processLogs) (92099) Got request for method 'glob' >Thread-18::DEBUG::2014-10-03 14:29:23,165::__init__::232::IOProcess::(_processLogs) (92040) Queuing response >Thread-18::DEBUG::2014-10-03 14:29:23,165::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 14:29:23,165::__init__::232::IOProcess::(_processLogs) (92100) Got request for method 'glob' >Thread-108377::WARNING::2014-10-03 14:29:33,290::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 14:29:33,291::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 14:29:33,291::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 14:29:43,292::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 14:29:43,292::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108378::WARNING::2014-10-03 14:30:43,292::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 14:30:43,293::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 14:30:43,293::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 14:30:53,293::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 14:30:53,294::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 14:30:53,294::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108379::WARNING::2014-10-03 14:31:53,294::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 14:31:53,295::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 14:31:53,295::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 14:32:03,296::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 14:32:03,296::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108380::WARNING::2014-10-03 14:33:03,296::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 14:33:03,297::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 14:33:03,297::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 14:33:13,298::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 14:33:13,298::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108381::WARNING::2014-10-03 14:34:13,298::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 14:34:13,298::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 14:34:13,298::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 14:34:23,300::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 14:34:23,300::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108382::WARNING::2014-10-03 14:35:23,300::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 14:35:23,300::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 14:35:23,300::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 14:35:33,301::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 14:35:33,302::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108383::WARNING::2014-10-03 14:36:33,301::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 14:36:33,302::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 14:36:33,302::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 14:36:43,302::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 14:36:43,303::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 14:36:43,303::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108384::WARNING::2014-10-03 14:37:43,303::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 14:37:43,304::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 14:37:43,304::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 14:37:53,305::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 14:37:53,305::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108385::WARNING::2014-10-03 14:38:53,305::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 14:38:53,306::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 14:38:53,306::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 14:39:03,307::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 14:39:03,307::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-03 14:39:27,324::__init__::232::IOProcess::(_processLogs) (92041) Queuing response >Thread-18::DEBUG::2014-10-03 14:39:27,325::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 14:39:27,325::__init__::232::IOProcess::(_processLogs) (92101) Got request for method 'glob' >Thread-18::DEBUG::2014-10-03 14:39:27,325::__init__::232::IOProcess::(_processLogs) (92042) Queuing response >Thread-18::DEBUG::2014-10-03 14:39:27,325::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 14:39:27,325::__init__::232::IOProcess::(_processLogs) (92102) Got request for method 'glob' >Thread-108386::WARNING::2014-10-03 14:40:03,307::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 14:40:03,307::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 14:40:03,308::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 14:40:13,309::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 14:40:13,309::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108387::WARNING::2014-10-03 14:41:13,309::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 14:41:13,309::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 14:41:13,309::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 14:41:23,311::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 14:41:23,311::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108388::WARNING::2014-10-03 14:42:23,310::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 14:42:23,311::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 14:42:23,311::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 14:42:33,312::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 14:42:33,313::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 14:42:33,313::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108389::WARNING::2014-10-03 14:43:33,313::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 14:43:33,314::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 14:43:33,315::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 14:43:43,316::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 14:43:43,316::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108390::WARNING::2014-10-03 14:44:43,316::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 14:44:43,317::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 14:44:43,317::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 14:44:53,318::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 14:44:53,319::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108391::WARNING::2014-10-03 14:45:53,318::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 14:45:53,319::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 14:45:53,319::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 14:46:03,320::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 14:46:03,321::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108392::WARNING::2014-10-03 14:47:03,320::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 14:47:03,321::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 14:47:03,321::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 14:47:13,322::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 14:47:13,323::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108393::WARNING::2014-10-03 14:48:13,322::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 14:48:13,323::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 14:48:13,323::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 14:48:23,324::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 14:48:23,325::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 14:48:23,325::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108394::WARNING::2014-10-03 14:49:23,325::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 14:49:23,325::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 14:49:23,325::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 14:49:31,484::__init__::232::IOProcess::(_processLogs) (92043) Queuing response >Thread-18::DEBUG::2014-10-03 14:49:31,485::__init__::232::IOProcess::(_processLogs) (92044) Queuing response >Thread-18::DEBUG::2014-10-03 14:49:31,485::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 14:49:31,485::__init__::232::IOProcess::(_processLogs) (92103) Got request for method 'glob' >Thread-18::DEBUG::2014-10-03 14:49:31,485::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 14:49:31,485::__init__::232::IOProcess::(_processLogs) (92104) Got request for method 'glob' >Thread-18::DEBUG::2014-10-03 14:49:33,327::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 14:49:33,327::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108395::WARNING::2014-10-03 14:50:33,327::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 14:50:33,327::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 14:50:33,328::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 14:50:43,329::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 14:50:43,329::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108396::WARNING::2014-10-03 14:51:43,329::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 14:51:43,329::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 14:51:43,329::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 14:51:53,331::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 14:51:53,331::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108397::WARNING::2014-10-03 14:52:53,330::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 14:52:53,331::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 14:52:53,331::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 14:53:03,332::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 14:53:03,333::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108398::WARNING::2014-10-03 14:54:03,332::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 14:54:03,333::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 14:54:03,333::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 14:54:13,333::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 14:54:13,334::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 14:54:13,335::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108399::WARNING::2014-10-03 14:55:13,334::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 14:55:13,335::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 14:55:13,335::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 14:55:23,336::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 14:55:23,337::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108400::WARNING::2014-10-03 14:56:23,336::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 14:56:23,337::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 14:56:23,337::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 14:56:33,338::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 14:56:33,339::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108401::WARNING::2014-10-03 14:57:33,338::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 14:57:33,339::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 14:57:33,339::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 14:57:43,340::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 14:57:43,340::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108402::WARNING::2014-10-03 14:58:43,340::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 14:58:43,341::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 14:58:43,341::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 14:58:53,342::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 14:58:53,342::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-03 14:59:35,645::__init__::232::IOProcess::(_processLogs) (92046) Queuing response >Thread-18::DEBUG::2014-10-03 14:59:35,645::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 14:59:35,645::__init__::232::IOProcess::(_processLogs) (92105) Got request for method 'glob' >Thread-18::DEBUG::2014-10-03 14:59:35,645::__init__::232::IOProcess::(_processLogs) (92045) Queuing response >Thread-18::DEBUG::2014-10-03 14:59:35,645::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 14:59:35,645::__init__::232::IOProcess::(_processLogs) (92106) Got request for method 'glob' >Thread-108403::WARNING::2014-10-03 14:59:53,342::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 14:59:53,343::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 14:59:53,343::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 15:00:03,343::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 15:00:03,344::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 15:00:03,344::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108404::WARNING::2014-10-03 15:01:03,344::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 15:01:03,345::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 15:01:03,345::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 15:01:13,346::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 15:01:13,346::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108405::WARNING::2014-10-03 15:02:13,346::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 15:02:13,347::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 15:02:13,347::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 15:02:23,348::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 15:02:23,348::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108406::WARNING::2014-10-03 15:03:23,348::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 15:03:23,348::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 15:03:23,349::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 15:03:33,350::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 15:03:33,350::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108407::WARNING::2014-10-03 15:04:33,350::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 15:04:33,350::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 15:04:33,350::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 15:04:43,352::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 15:04:43,352::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108408::WARNING::2014-10-03 15:05:43,351::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 15:05:43,352::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 15:05:43,352::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 15:05:53,353::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 15:05:53,354::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 15:05:53,354::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108409::WARNING::2014-10-03 15:06:53,353::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 15:06:53,354::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 15:06:53,354::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 15:07:03,355::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 15:07:03,356::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108410::WARNING::2014-10-03 15:08:03,355::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 15:08:03,356::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 15:08:03,356::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 15:08:13,357::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 15:08:13,358::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108411::WARNING::2014-10-03 15:09:13,357::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 15:09:13,358::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 15:09:13,358::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 15:09:23,359::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 15:09:23,360::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-03 15:09:39,804::__init__::232::IOProcess::(_processLogs) (92047) Queuing response >Thread-18::DEBUG::2014-10-03 15:09:39,805::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 15:09:39,805::__init__::232::IOProcess::(_processLogs) (92107) Got request for method 'glob' >Thread-108412::WARNING::2014-10-03 15:10:23,359::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 15:10:23,360::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 15:10:23,360::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 15:10:33,361::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 15:10:33,362::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108413::WARNING::2014-10-03 15:11:33,361::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 15:11:33,362::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 15:11:33,362::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 15:11:43,362::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 15:11:43,363::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 15:11:43,364::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108414::WARNING::2014-10-03 15:12:43,363::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 15:12:43,364::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 15:12:43,364::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 15:12:53,365::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 15:12:53,366::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108415::WARNING::2014-10-03 15:13:53,365::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 15:13:53,366::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 15:13:53,366::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 15:14:03,367::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 15:14:03,368::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108416::WARNING::2014-10-03 15:15:03,367::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 15:15:03,368::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 15:15:03,368::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 15:15:13,369::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 15:15:13,370::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108417::WARNING::2014-10-03 15:16:13,369::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 15:16:13,370::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 15:16:13,370::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 15:16:23,371::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 15:16:23,372::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108418::WARNING::2014-10-03 15:17:23,372::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 15:17:23,372::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 15:17:23,372::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 15:17:33,373::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 15:17:33,374::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 15:17:33,374::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108419::WARNING::2014-10-03 15:18:33,374::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 15:18:33,374::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 15:18:33,374::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 15:18:43,376::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 15:18:43,376::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108420::WARNING::2014-10-03 15:19:43,376::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 15:19:43,376::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 15:19:43,376::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 15:19:43,964::__init__::232::IOProcess::(_processLogs) (92048) Queuing response >Thread-18::DEBUG::2014-10-03 15:19:43,964::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 15:19:43,965::__init__::232::IOProcess::(_processLogs) (92108) Got request for method 'glob' >Thread-18::DEBUG::2014-10-03 15:19:43,965::__init__::232::IOProcess::(_processLogs) (92049) Queuing response >Thread-18::DEBUG::2014-10-03 15:19:43,965::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 15:19:43,965::__init__::232::IOProcess::(_processLogs) (92050) Queuing response >Thread-18::DEBUG::2014-10-03 15:19:43,965::__init__::232::IOProcess::(_processLogs) (92109) Got request for method 'glob' >Thread-18::DEBUG::2014-10-03 15:19:43,965::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 15:19:43,965::__init__::232::IOProcess::(_processLogs) (92110) Got request for method 'glob' >Thread-18::DEBUG::2014-10-03 15:19:53,378::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 15:19:53,378::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108421::WARNING::2014-10-03 15:20:53,378::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 15:20:53,378::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 15:20:53,378::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 15:21:03,379::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 15:21:03,380::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108422::WARNING::2014-10-03 15:22:03,379::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 15:22:03,380::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 15:22:03,380::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 15:22:13,381::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 15:22:13,381::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108423::WARNING::2014-10-03 15:23:13,381::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 15:23:13,382::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 15:23:13,382::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 15:23:23,382::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 15:23:23,383::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 15:23:23,383::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108424::WARNING::2014-10-03 15:24:23,383::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 15:24:23,383::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 15:24:23,384::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 15:24:33,385::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 15:24:33,385::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108425::WARNING::2014-10-03 15:25:33,385::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 15:25:33,385::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 15:25:33,385::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 15:25:43,387::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 15:25:43,387::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108426::WARNING::2014-10-03 15:26:43,386::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 15:26:43,387::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 15:26:43,387::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 15:26:53,388::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 15:26:53,388::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108427::WARNING::2014-10-03 15:27:53,388::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 15:27:53,389::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 15:27:53,389::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 15:28:03,390::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 15:28:03,390::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108428::WARNING::2014-10-03 15:29:03,390::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 15:29:03,390::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 15:29:03,391::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 15:29:13,391::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 15:29:13,392::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 15:29:13,392::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-03 15:29:48,124::__init__::232::IOProcess::(_processLogs) (92051) Queuing response >Thread-18::DEBUG::2014-10-03 15:29:48,125::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 15:29:48,125::__init__::232::IOProcess::(_processLogs) (92111) Got request for method 'glob' >Thread-18::DEBUG::2014-10-03 15:29:48,125::__init__::232::IOProcess::(_processLogs) (92052) Queuing response >Thread-18::DEBUG::2014-10-03 15:29:48,125::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 15:29:48,125::__init__::232::IOProcess::(_processLogs) (92112) Got request for method 'glob' >Thread-108429::WARNING::2014-10-03 15:30:13,392::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 15:30:13,392::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 15:30:13,392::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 15:30:23,394::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 15:30:23,394::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108430::WARNING::2014-10-03 15:31:23,394::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 15:31:23,394::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 15:31:23,394::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 15:31:33,395::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 15:31:33,395::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108431::WARNING::2014-10-03 15:32:33,395::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 15:32:33,396::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 15:32:33,396::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 15:32:43,397::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 15:32:43,397::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108432::WARNING::2014-10-03 15:33:43,397::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 15:33:43,398::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 15:33:43,398::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 15:33:53,399::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 15:33:53,399::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108433::WARNING::2014-10-03 15:34:53,399::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 15:34:53,400::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 15:34:53,400::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 15:35:03,400::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 15:35:03,401::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 15:35:03,401::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108434::WARNING::2014-10-03 15:36:03,401::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 15:36:03,402::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 15:36:03,402::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 15:36:13,403::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 15:36:13,403::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108435::WARNING::2014-10-03 15:37:13,403::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 15:37:13,404::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 15:37:13,404::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 15:37:23,405::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 15:37:23,405::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108436::WARNING::2014-10-03 15:38:23,405::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 15:38:23,406::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 15:38:23,407::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 15:38:33,409::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 15:38:33,409::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108437::WARNING::2014-10-03 15:39:33,409::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 15:39:33,409::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 15:39:33,410::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 15:39:43,411::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 15:39:43,411::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-03 15:39:54,332::__init__::232::IOProcess::(_processLogs) (92053) Queuing response >Thread-18::DEBUG::2014-10-03 15:39:54,333::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 15:39:54,333::__init__::232::IOProcess::(_processLogs) (92113) Got request for method 'glob' >Thread-18::DEBUG::2014-10-03 15:39:54,333::__init__::232::IOProcess::(_processLogs) (92054) Queuing response >Thread-18::DEBUG::2014-10-03 15:39:54,333::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 15:39:54,333::__init__::232::IOProcess::(_processLogs) (92114) Got request for method 'glob' >Thread-108438::WARNING::2014-10-03 15:40:43,411::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 15:40:43,411::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 15:40:43,412::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 15:40:53,412::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 15:40:53,413::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 15:40:53,413::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108439::WARNING::2014-10-03 15:41:53,413::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 15:41:53,413::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 15:41:53,414::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 15:42:03,415::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 15:42:03,415::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108440::WARNING::2014-10-03 15:43:03,415::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 15:43:03,415::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 15:43:03,415::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 15:43:13,417::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 15:43:13,417::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108441::WARNING::2014-10-03 15:44:13,417::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 15:44:13,417::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 15:44:13,417::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 15:44:23,419::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 15:44:23,419::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108442::WARNING::2014-10-03 15:45:23,418::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 15:45:23,419::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 15:45:23,419::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 15:45:33,421::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 15:45:33,421::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108443::WARNING::2014-10-03 15:46:33,421::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 15:46:33,421::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 15:46:33,421::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 15:46:43,422::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 15:46:43,423::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 15:46:43,423::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108444::WARNING::2014-10-03 15:47:43,423::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 15:47:43,423::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 15:47:43,423::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 15:47:53,425::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 15:47:53,426::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108445::WARNING::2014-10-03 15:48:53,425::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 15:48:53,426::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 15:48:53,426::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 15:49:03,427::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 15:49:03,428::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-03 15:49:58,492::__init__::232::IOProcess::(_processLogs) (92055) Queuing response >Thread-18::DEBUG::2014-10-03 15:49:58,493::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 15:49:58,493::__init__::232::IOProcess::(_processLogs) (92115) Got request for method 'glob' >Thread-18::DEBUG::2014-10-03 15:49:58,493::__init__::232::IOProcess::(_processLogs) (92056) Queuing response >Thread-18::DEBUG::2014-10-03 15:49:58,493::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 15:49:58,493::__init__::232::IOProcess::(_processLogs) (92116) Got request for method 'glob' >Thread-108446::WARNING::2014-10-03 15:50:03,427::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 15:50:03,428::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 15:50:03,428::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 15:50:13,429::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 15:50:13,430::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108447::WARNING::2014-10-03 15:51:13,429::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 15:51:13,430::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 15:51:13,430::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 15:51:23,431::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 15:51:23,431::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108448::WARNING::2014-10-03 15:52:23,431::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 15:52:23,432::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 15:52:23,432::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 15:52:33,432::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 15:52:33,433::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 15:52:33,433::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108449::WARNING::2014-10-03 15:53:33,433::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 15:53:33,434::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 15:53:33,434::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 15:53:43,435::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 15:53:43,435::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108450::WARNING::2014-10-03 15:54:43,435::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 15:54:43,436::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 15:54:43,436::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 15:54:53,437::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 15:54:53,437::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108451::WARNING::2014-10-03 15:55:53,437::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 15:55:53,437::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 15:55:53,438::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 15:56:03,439::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 15:56:03,439::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108452::WARNING::2014-10-03 15:57:03,439::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 15:57:03,439::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 15:57:03,440::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 15:57:13,441::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 15:57:13,441::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108453::WARNING::2014-10-03 15:58:13,441::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 15:58:13,441::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 15:58:13,441::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 15:58:23,442::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 15:58:23,443::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 15:58:23,443::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108454::WARNING::2014-10-03 15:59:23,443::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 15:59:23,443::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 15:59:23,443::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 15:59:33,444::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 15:59:33,445::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-03 16:00:02,652::__init__::232::IOProcess::(_processLogs) (92057) Queuing response >Thread-18::DEBUG::2014-10-03 16:00:02,653::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 16:00:02,653::__init__::232::IOProcess::(_processLogs) (92117) Got request for method 'glob' >Thread-108455::WARNING::2014-10-03 16:00:33,444::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 16:00:33,445::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 16:00:33,445::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 16:00:43,446::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 16:00:43,447::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108456::WARNING::2014-10-03 16:01:43,446::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 16:01:43,447::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 16:01:43,447::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 16:01:53,448::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 16:01:53,448::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108457::WARNING::2014-10-03 16:02:53,448::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 16:02:53,449::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 16:02:53,449::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 16:03:03,450::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 16:03:03,450::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108458::WARNING::2014-10-03 16:04:03,450::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 16:04:03,451::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 16:04:03,451::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 16:04:13,451::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 16:04:13,452::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 16:04:13,452::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108459::WARNING::2014-10-03 16:05:13,452::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 16:05:13,453::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 16:05:13,453::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 16:05:23,454::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 16:05:23,454::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108460::WARNING::2014-10-03 16:06:23,454::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 16:06:23,455::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 16:06:23,455::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 16:06:33,456::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 16:06:33,456::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108461::WARNING::2014-10-03 16:07:33,456::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 16:07:33,457::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 16:07:33,457::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 16:07:43,458::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 16:07:43,458::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108462::WARNING::2014-10-03 16:08:43,458::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 16:08:43,459::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 16:08:43,459::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 16:08:53,460::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 16:08:53,460::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108463::WARNING::2014-10-03 16:09:53,460::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 16:09:53,460::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 16:09:53,461::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 16:10:03,461::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 16:10:03,462::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 16:10:03,462::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-03 16:10:06,812::__init__::232::IOProcess::(_processLogs) (92058) Queuing response >Thread-18::DEBUG::2014-10-03 16:10:06,813::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 16:10:06,813::__init__::232::IOProcess::(_processLogs) (92118) Got request for method 'glob' >Thread-108464::WARNING::2014-10-03 16:11:03,462::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 16:11:03,462::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 16:11:03,463::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 16:11:13,464::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 16:11:13,464::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108465::WARNING::2014-10-03 16:12:13,464::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 16:12:13,464::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 16:12:13,464::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 16:12:23,466::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 16:12:23,466::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108466::WARNING::2014-10-03 16:13:23,466::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 16:13:23,466::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 16:13:23,466::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 16:13:33,467::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 16:13:33,468::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108467::WARNING::2014-10-03 16:14:33,467::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 16:14:33,468::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 16:14:33,468::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 16:14:43,469::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 16:14:43,469::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108468::WARNING::2014-10-03 16:15:43,469::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 16:15:43,470::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 16:15:43,470::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 16:15:53,470::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 16:15:53,471::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 16:15:53,471::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108469::WARNING::2014-10-03 16:16:53,471::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 16:16:53,472::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 16:16:53,472::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 16:17:03,474::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 16:17:03,474::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108470::WARNING::2014-10-03 16:18:03,473::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 16:18:03,474::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 16:18:03,474::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 16:18:13,475::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 16:18:13,476::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108471::WARNING::2014-10-03 16:19:13,475::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 16:19:13,476::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 16:19:13,476::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 16:19:23,477::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 16:19:23,478::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-03 16:20:10,972::__init__::232::IOProcess::(_processLogs) (92059) Queuing response >Thread-18::DEBUG::2014-10-03 16:20:10,973::__init__::232::IOProcess::(_processLogs) (92060) Queuing response >Thread-18::DEBUG::2014-10-03 16:20:10,973::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 16:20:10,973::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 16:20:10,973::__init__::232::IOProcess::(_processLogs) (92119) Got request for method 'glob' >Thread-18::DEBUG::2014-10-03 16:20:10,973::__init__::232::IOProcess::(_processLogs) (92120) Got request for method 'glob' >Thread-108472::WARNING::2014-10-03 16:20:23,477::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 16:20:23,478::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 16:20:23,478::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 16:20:33,479::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 16:20:33,480::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108473::WARNING::2014-10-03 16:21:33,479::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 16:21:33,480::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 16:21:33,480::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 16:21:43,480::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 16:21:43,481::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 16:21:43,481::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108474::WARNING::2014-10-03 16:22:43,481::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 16:22:43,482::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 16:22:43,482::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 16:22:53,483::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 16:22:53,483::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108475::WARNING::2014-10-03 16:23:53,483::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 16:23:53,483::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 16:23:53,484::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 16:24:03,485::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 16:24:03,485::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108476::WARNING::2014-10-03 16:25:03,485::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 16:25:03,485::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 16:25:03,485::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 16:25:13,487::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 16:25:13,487::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108477::WARNING::2014-10-03 16:26:13,487::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 16:26:13,487::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 16:26:13,487::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 16:26:23,488::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 16:26:23,488::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108478::WARNING::2014-10-03 16:27:23,488::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 16:27:23,489::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 16:27:23,489::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 16:27:33,489::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 16:27:33,490::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 16:27:33,490::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108479::WARNING::2014-10-03 16:28:33,490::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 16:28:33,491::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 16:28:33,491::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 16:28:43,492::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 16:28:43,492::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108480::WARNING::2014-10-03 16:29:43,492::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 16:29:43,492::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 16:29:43,493::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 16:29:53,494::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 16:29:53,494::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-03 16:30:15,132::__init__::232::IOProcess::(_processLogs) (92062) Queuing response >Thread-18::DEBUG::2014-10-03 16:30:15,133::__init__::232::IOProcess::(_processLogs) (92061) Queuing response >Thread-18::DEBUG::2014-10-03 16:30:15,133::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 16:30:15,133::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 16:30:15,133::__init__::232::IOProcess::(_processLogs) (92122) Got request for method 'glob' >Thread-18::DEBUG::2014-10-03 16:30:15,133::__init__::232::IOProcess::(_processLogs) (92121) Got request for method 'glob' >Thread-108481::WARNING::2014-10-03 16:30:53,494::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 16:30:53,494::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 16:30:53,494::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 16:31:03,495::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 16:31:03,496::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108482::WARNING::2014-10-03 16:32:03,495::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 16:32:03,496::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 16:32:03,496::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 16:32:13,497::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 16:32:13,497::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108483::WARNING::2014-10-03 16:33:13,497::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 16:33:13,497::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 16:33:13,498::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 16:33:23,498::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 16:33:23,499::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 16:33:23,499::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108484::WARNING::2014-10-03 16:34:23,499::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 16:34:23,499::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 16:34:23,500::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 16:34:33,501::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 16:34:33,501::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108485::WARNING::2014-10-03 16:35:33,501::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 16:35:33,501::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 16:35:33,502::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 16:35:43,503::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 16:35:43,503::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108486::WARNING::2014-10-03 16:36:43,503::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 16:36:43,503::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 16:36:43,503::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 16:36:53,505::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 16:36:53,505::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108487::WARNING::2014-10-03 16:37:53,504::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 16:37:53,505::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 16:37:53,505::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 16:38:03,506::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 16:38:03,507::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108488::WARNING::2014-10-03 16:39:03,506::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 16:39:03,507::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 16:39:03,507::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 16:39:13,508::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 16:39:13,509::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 16:39:13,509::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108489::WARNING::2014-10-03 16:40:13,508::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 16:40:13,509::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 16:40:13,509::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 16:40:19,292::__init__::232::IOProcess::(_processLogs) (92063) Queuing response >Thread-18::DEBUG::2014-10-03 16:40:19,293::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 16:40:19,293::__init__::232::IOProcess::(_processLogs) (92123) Got request for method 'glob' >Thread-18::DEBUG::2014-10-03 16:40:19,293::__init__::232::IOProcess::(_processLogs) (92064) Queuing response >Thread-18::DEBUG::2014-10-03 16:40:19,293::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 16:40:19,293::__init__::232::IOProcess::(_processLogs) (92124) Got request for method 'glob' >Thread-18::DEBUG::2014-10-03 16:40:23,511::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 16:40:23,511::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108490::WARNING::2014-10-03 16:41:23,511::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 16:41:23,511::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 16:41:23,511::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 16:41:33,513::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 16:41:33,513::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108491::WARNING::2014-10-03 16:42:33,512::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 16:42:33,513::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 16:42:33,513::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 16:42:43,514::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 16:42:43,515::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108492::WARNING::2014-10-03 16:43:43,514::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 16:43:43,515::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 16:43:43,515::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 16:43:53,516::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 16:43:53,516::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108493::WARNING::2014-10-03 16:44:53,516::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 16:44:53,517::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 16:44:53,517::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 16:45:03,517::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 16:45:03,518::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 16:45:03,519::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108494::WARNING::2014-10-03 16:46:03,518::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 16:46:03,519::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 16:46:03,519::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 16:46:13,520::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 16:46:13,520::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108495::WARNING::2014-10-03 16:47:13,520::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 16:47:13,521::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 16:47:13,521::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 16:47:23,522::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 16:47:23,522::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108496::WARNING::2014-10-03 16:48:23,522::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 16:48:23,523::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 16:48:23,523::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 16:48:33,524::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 16:48:33,524::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108497::WARNING::2014-10-03 16:49:33,524::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 16:49:33,525::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 16:49:33,525::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 16:49:43,526::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 16:49:43,526::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-03 16:50:23,452::__init__::232::IOProcess::(_processLogs) (92065) Queuing response >Thread-18::DEBUG::2014-10-03 16:50:23,453::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 16:50:23,453::__init__::232::IOProcess::(_processLogs) (92125) Got request for method 'glob' >Thread-108498::WARNING::2014-10-03 16:50:43,526::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 16:50:43,527::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 16:50:43,527::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 16:50:53,527::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 16:50:53,528::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 16:50:53,529::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108499::WARNING::2014-10-03 16:51:53,528::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 16:51:53,529::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 16:51:53,529::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 16:52:03,530::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 16:52:03,530::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108500::WARNING::2014-10-03 16:53:03,530::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 16:53:03,531::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 16:53:03,531::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 16:53:13,532::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 16:53:13,532::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108501::WARNING::2014-10-03 16:54:13,532::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 16:54:13,533::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 16:54:13,533::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 16:54:23,534::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 16:54:23,534::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108502::WARNING::2014-10-03 16:55:23,534::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 16:55:23,534::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 16:55:23,535::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 16:55:33,536::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 16:55:33,536::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108503::WARNING::2014-10-03 16:56:33,536::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 16:56:33,536::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 16:56:33,536::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 16:56:43,537::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 16:56:43,538::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 16:56:43,538::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108504::WARNING::2014-10-03 16:57:43,538::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 16:57:43,538::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 16:57:43,538::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 16:57:53,540::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 16:57:53,540::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108505::WARNING::2014-10-03 16:58:53,540::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 16:58:53,540::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 16:58:53,540::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 16:59:03,541::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 16:59:03,542::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108506::WARNING::2014-10-03 17:00:03,541::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 17:00:03,542::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 17:00:03,542::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 17:00:13,543::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 17:00:13,544::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-03 17:00:27,612::__init__::232::IOProcess::(_processLogs) (92066) Queuing response >Thread-18::DEBUG::2014-10-03 17:00:27,613::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 17:00:27,613::__init__::232::IOProcess::(_processLogs) (92126) Got request for method 'glob' >Thread-108507::WARNING::2014-10-03 17:01:13,543::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 17:01:13,544::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 17:01:13,544::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 17:01:23,545::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 17:01:23,546::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108508::WARNING::2014-10-03 17:02:23,545::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 17:02:23,546::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 17:02:23,546::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 17:02:33,546::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 17:02:33,547::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 17:02:33,548::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108509::WARNING::2014-10-03 17:03:33,547::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 17:03:33,548::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 17:03:33,548::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 17:03:43,549::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 17:03:43,549::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108510::WARNING::2014-10-03 17:04:43,549::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 17:04:43,550::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 17:04:43,550::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 17:04:53,551::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 17:04:53,551::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108511::WARNING::2014-10-03 17:05:53,551::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 17:05:53,551::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 17:05:53,552::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 17:06:03,553::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 17:06:03,553::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108512::WARNING::2014-10-03 17:07:03,553::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 17:07:03,553::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 17:07:03,554::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 17:07:13,555::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 17:07:13,555::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108513::WARNING::2014-10-03 17:08:13,555::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 17:08:13,555::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 17:08:13,555::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 17:08:23,556::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 17:08:23,557::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 17:08:23,557::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108514::WARNING::2014-10-03 17:09:23,557::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 17:09:23,557::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 17:09:23,557::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 17:09:33,559::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 17:09:33,559::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-03 17:10:31,772::__init__::232::IOProcess::(_processLogs) (92067) Queuing response >Thread-18::DEBUG::2014-10-03 17:10:31,773::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 17:10:31,773::__init__::232::IOProcess::(_processLogs) (92127) Got request for method 'glob' >Thread-18::DEBUG::2014-10-03 17:10:31,773::__init__::232::IOProcess::(_processLogs) (92068) Queuing response >Thread-18::DEBUG::2014-10-03 17:10:31,773::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 17:10:31,773::__init__::232::IOProcess::(_processLogs) (92128) Got request for method 'glob' >Thread-108515::WARNING::2014-10-03 17:10:33,559::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 17:10:33,559::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 17:10:33,559::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 17:10:43,561::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 17:10:43,561::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108516::WARNING::2014-10-03 17:11:43,560::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 17:11:43,561::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 17:11:43,561::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 17:11:53,562::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 17:11:53,563::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108517::WARNING::2014-10-03 17:12:53,562::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 17:12:53,563::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 17:12:53,563::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 17:13:03,564::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 17:13:03,564::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108518::WARNING::2014-10-03 17:14:03,564::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 17:14:03,565::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 17:14:03,565::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 17:14:13,565::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 17:14:13,566::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 17:14:13,566::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108519::WARNING::2014-10-03 17:15:13,566::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 17:15:13,567::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 17:15:13,567::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 17:15:23,568::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 17:15:23,568::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108520::WARNING::2014-10-03 17:16:23,568::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 17:16:23,569::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 17:16:23,569::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 17:16:33,570::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 17:16:33,570::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108521::WARNING::2014-10-03 17:17:33,570::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 17:17:33,571::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 17:17:33,571::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 17:17:43,572::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 17:17:43,572::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108522::WARNING::2014-10-03 17:18:43,572::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 17:18:43,572::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 17:18:43,573::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 17:18:53,574::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 17:18:53,574::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108523::WARNING::2014-10-03 17:19:53,574::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 17:19:53,574::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 17:19:53,575::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 17:20:03,575::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 17:20:03,576::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 17:20:03,576::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-03 17:20:37,980::__init__::232::IOProcess::(_processLogs) (92069) Queuing response >Thread-18::DEBUG::2014-10-03 17:20:37,981::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 17:20:37,981::__init__::232::IOProcess::(_processLogs) (92129) Got request for method 'glob' >Thread-18::DEBUG::2014-10-03 17:20:37,981::__init__::232::IOProcess::(_processLogs) (92070) Queuing response >Thread-18::DEBUG::2014-10-03 17:20:37,981::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 17:20:37,981::__init__::232::IOProcess::(_processLogs) (92130) Got request for method 'glob' >Thread-108524::WARNING::2014-10-03 17:21:03,576::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 17:21:03,577::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 17:21:03,577::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 17:21:13,580::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 17:21:13,580::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108525::WARNING::2014-10-03 17:22:13,580::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 17:22:13,580::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 17:22:13,581::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 17:22:23,582::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 17:22:23,582::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108526::WARNING::2014-10-03 17:23:23,582::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 17:23:23,582::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 17:23:23,582::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 17:23:33,584::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 17:23:33,584::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108527::WARNING::2014-10-03 17:24:33,583::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 17:24:33,584::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 17:24:33,584::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 17:24:43,585::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 17:24:43,585::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108528::WARNING::2014-10-03 17:25:43,585::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 17:25:43,586::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 17:25:43,586::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 17:25:53,586::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 17:25:53,587::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 17:25:53,587::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108529::WARNING::2014-10-03 17:26:53,587::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 17:26:53,588::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 17:26:53,588::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 17:27:03,589::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 17:27:03,590::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108530::WARNING::2014-10-03 17:28:03,589::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 17:28:03,590::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 17:28:03,590::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 17:28:13,591::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 17:28:13,591::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108531::WARNING::2014-10-03 17:29:13,591::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 17:29:13,592::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 17:29:13,592::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 17:29:23,593::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 17:29:23,593::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108532::WARNING::2014-10-03 17:30:23,593::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 17:30:23,594::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 17:30:23,594::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 17:30:33,595::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 17:30:33,595::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-03 17:30:42,140::__init__::232::IOProcess::(_processLogs) (92071) Queuing response >Thread-18::DEBUG::2014-10-03 17:30:42,141::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 17:30:42,141::__init__::232::IOProcess::(_processLogs) (92131) Got request for method 'glob' >Thread-18::DEBUG::2014-10-03 17:30:42,141::__init__::232::IOProcess::(_processLogs) (92072) Queuing response >Thread-18::DEBUG::2014-10-03 17:30:42,141::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 17:30:42,141::__init__::232::IOProcess::(_processLogs) (92132) Got request for method 'glob' >Thread-108533::WARNING::2014-10-03 17:31:33,595::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 17:31:33,595::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 17:31:33,596::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 17:31:43,596::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 17:31:43,597::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 17:31:43,597::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108534::WARNING::2014-10-03 17:32:43,597::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 17:32:43,597::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 17:32:43,598::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 17:32:53,599::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 17:32:53,599::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108535::WARNING::2014-10-03 17:33:53,599::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 17:33:53,599::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 17:33:53,599::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 17:34:03,601::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 17:34:03,601::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108536::WARNING::2014-10-03 17:35:03,601::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 17:35:03,601::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 17:35:03,601::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 17:35:13,603::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 17:35:13,603::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108537::WARNING::2014-10-03 17:36:13,602::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 17:36:13,603::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 17:36:13,603::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 17:36:23,604::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 17:36:23,605::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108538::WARNING::2014-10-03 17:37:23,604::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 17:37:23,605::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 17:37:23,605::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 17:37:33,605::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 17:37:33,606::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 17:37:33,607::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108539::WARNING::2014-10-03 17:38:33,606::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 17:38:33,607::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 17:38:33,607::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 17:38:43,608::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 17:38:43,609::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108540::WARNING::2014-10-03 17:39:43,608::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 17:39:43,609::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 17:39:43,609::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 17:39:53,610::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 17:39:53,610::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-03 17:40:46,300::__init__::232::IOProcess::(_processLogs) (92073) Queuing response >Thread-18::DEBUG::2014-10-03 17:40:46,301::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 17:40:46,301::__init__::232::IOProcess::(_processLogs) (92133) Got request for method 'glob' >Thread-18::DEBUG::2014-10-03 17:40:46,301::__init__::232::IOProcess::(_processLogs) (92074) Queuing response >Thread-18::DEBUG::2014-10-03 17:40:46,301::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 17:40:46,301::__init__::232::IOProcess::(_processLogs) (92134) Got request for method 'glob' >Thread-108541::WARNING::2014-10-03 17:40:53,610::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 17:40:53,611::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 17:40:53,611::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 17:41:03,612::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 17:41:03,612::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108542::WARNING::2014-10-03 17:42:03,612::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 17:42:03,613::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 17:42:03,613::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 17:42:13,614::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 17:42:13,614::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108543::WARNING::2014-10-03 17:43:13,614::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 17:43:13,615::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 17:43:13,615::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 17:43:23,615::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 17:43:23,616::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 17:43:23,616::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108544::WARNING::2014-10-03 17:44:23,616::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 17:44:23,617::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 17:44:23,617::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 17:44:33,618::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 17:44:33,618::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108545::WARNING::2014-10-03 17:45:33,618::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 17:45:33,618::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 17:45:33,619::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 17:45:43,620::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 17:45:43,620::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108546::WARNING::2014-10-03 17:46:43,620::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 17:46:43,620::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 17:46:43,620::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 17:46:53,622::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 17:46:53,622::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108547::WARNING::2014-10-03 17:47:53,622::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 17:47:53,622::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 17:47:53,622::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 17:48:03,623::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 17:48:03,624::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108548::WARNING::2014-10-03 17:49:03,623::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 17:49:03,624::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 17:49:03,624::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 17:49:13,625::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 17:49:13,626::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 17:49:13,626::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108549::WARNING::2014-10-03 17:50:13,625::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 17:50:13,626::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 17:50:13,626::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 17:50:23,627::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 17:50:23,628::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-03 17:50:52,508::__init__::232::IOProcess::(_processLogs) (92075) Queuing response >Thread-18::DEBUG::2014-10-03 17:50:52,509::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 17:50:52,509::__init__::232::IOProcess::(_processLogs) (92135) Got request for method 'glob' >Thread-18::DEBUG::2014-10-03 17:50:52,509::__init__::232::IOProcess::(_processLogs) (92076) Queuing response >Thread-18::DEBUG::2014-10-03 17:50:52,509::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 17:50:52,509::__init__::232::IOProcess::(_processLogs) (92136) Got request for method 'glob' >Thread-108550::WARNING::2014-10-03 17:51:23,627::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 17:51:23,628::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 17:51:23,628::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 17:51:33,629::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 17:51:33,630::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108551::WARNING::2014-10-03 17:52:33,629::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 17:52:33,630::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 17:52:33,630::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 17:52:43,631::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 17:52:43,631::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108552::WARNING::2014-10-03 17:53:43,631::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 17:53:43,632::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 17:53:43,632::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 17:53:53,633::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 17:53:53,634::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108553::WARNING::2014-10-03 17:54:53,633::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 17:54:53,634::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 17:54:53,634::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 17:55:03,634::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 17:55:03,635::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 17:55:03,636::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108554::WARNING::2014-10-03 17:56:03,635::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 17:56:03,636::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 17:56:03,636::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 17:56:13,637::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 17:56:13,637::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108555::WARNING::2014-10-03 17:57:13,637::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 17:57:13,638::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 17:57:13,638::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 17:57:23,639::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 17:57:23,639::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108556::WARNING::2014-10-03 17:58:23,639::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 17:58:23,639::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 17:58:23,640::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 17:58:33,641::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 17:58:33,641::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108557::WARNING::2014-10-03 17:59:33,641::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 17:59:33,641::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 17:59:33,642::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 17:59:43,643::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 17:59:43,643::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108558::WARNING::2014-10-03 18:00:43,643::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 18:00:43,643::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 18:00:43,643::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 18:00:53,644::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 18:00:53,645::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 18:00:53,645::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-03 18:00:56,668::__init__::232::IOProcess::(_processLogs) (92077) Queuing response >Thread-18::DEBUG::2014-10-03 18:00:56,669::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 18:00:56,669::__init__::232::IOProcess::(_processLogs) (92137) Got request for method 'glob' >Thread-18::DEBUG::2014-10-03 18:00:56,669::__init__::232::IOProcess::(_processLogs) (92078) Queuing response >Thread-18::DEBUG::2014-10-03 18:00:56,669::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 18:00:56,669::__init__::232::IOProcess::(_processLogs) (92138) Got request for method 'glob' >Thread-108559::WARNING::2014-10-03 18:01:53,645::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 18:01:53,645::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 18:01:53,645::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 18:02:03,647::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 18:02:03,647::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108560::WARNING::2014-10-03 18:03:03,647::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 18:03:03,647::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 18:03:03,647::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 18:03:13,648::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 18:03:13,649::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108561::WARNING::2014-10-03 18:04:13,648::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 18:04:13,649::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 18:04:13,649::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 18:04:23,650::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 18:04:23,651::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108562::WARNING::2014-10-03 18:05:23,650::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 18:05:23,651::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 18:05:23,651::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 18:05:33,652::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 18:05:33,653::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108563::WARNING::2014-10-03 18:06:33,652::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 18:06:33,653::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 18:06:33,653::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 18:06:43,654::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 18:06:43,655::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 18:06:43,655::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108564::WARNING::2014-10-03 18:07:43,654::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 18:07:43,655::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 18:07:43,655::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 18:07:53,656::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 18:07:53,657::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108565::WARNING::2014-10-03 18:08:53,657::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 18:08:53,657::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 18:08:53,657::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 18:09:03,659::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 18:09:03,659::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108566::WARNING::2014-10-03 18:10:03,658::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 18:10:03,659::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 18:10:03,659::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 18:10:13,661::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 18:10:13,661::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-03 18:11:02,876::__init__::232::IOProcess::(_processLogs) (92079) Queuing response >Thread-18::DEBUG::2014-10-03 18:11:02,877::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 18:11:02,877::__init__::232::IOProcess::(_processLogs) (92139) Got request for method 'glob' >Thread-18::DEBUG::2014-10-03 18:11:02,877::__init__::232::IOProcess::(_processLogs) (92080) Queuing response >Thread-18::DEBUG::2014-10-03 18:11:02,877::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 18:11:02,877::__init__::232::IOProcess::(_processLogs) (92140) Got request for method 'glob' >Thread-108567::WARNING::2014-10-03 18:11:13,660::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 18:11:13,661::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 18:11:13,661::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 18:11:23,662::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 18:11:23,663::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108568::WARNING::2014-10-03 18:12:23,662::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 18:12:23,663::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 18:12:23,663::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 18:12:33,664::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 18:12:33,665::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 18:12:33,665::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108569::WARNING::2014-10-03 18:13:33,664::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 18:13:33,665::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 18:13:33,665::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 18:13:43,666::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 18:13:43,667::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108570::WARNING::2014-10-03 18:14:43,666::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 18:14:43,667::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 18:14:43,667::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 18:14:53,668::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 18:14:53,669::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108571::WARNING::2014-10-03 18:15:53,668::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 18:15:53,669::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 18:15:53,669::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 18:16:03,670::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 18:16:03,670::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108572::WARNING::2014-10-03 18:17:03,670::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 18:17:03,671::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 18:17:03,671::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 18:17:13,672::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 18:17:13,672::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108573::WARNING::2014-10-03 18:18:13,672::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 18:18:13,673::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 18:18:13,673::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 18:18:23,674::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 18:18:23,675::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 18:18:23,675::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108574::WARNING::2014-10-03 18:19:23,675::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 18:19:23,675::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 18:19:23,675::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 18:19:33,677::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 18:19:33,677::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108575::WARNING::2014-10-03 18:20:33,677::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 18:20:33,677::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 18:20:33,677::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 18:20:43,679::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 18:20:43,679::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-03 18:21:07,036::__init__::232::IOProcess::(_processLogs) (92082) Queuing response >Thread-18::DEBUG::2014-10-03 18:21:07,037::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 18:21:07,037::__init__::232::IOProcess::(_processLogs) (92141) Got request for method 'glob' >Thread-18::DEBUG::2014-10-03 18:21:07,037::__init__::232::IOProcess::(_processLogs) (92081) Queuing response >Thread-18::DEBUG::2014-10-03 18:21:07,037::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 18:21:07,037::__init__::232::IOProcess::(_processLogs) (92142) Got request for method 'glob' >Thread-108576::WARNING::2014-10-03 18:21:43,679::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 18:21:43,679::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 18:21:43,679::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 18:21:53,680::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 18:21:53,681::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108577::WARNING::2014-10-03 18:22:53,680::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 18:22:53,681::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 18:22:53,681::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 18:23:03,682::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 18:23:03,682::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108578::WARNING::2014-10-03 18:24:03,682::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 18:24:03,683::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 18:24:03,684::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 18:24:13,685::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 18:24:13,686::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 18:24:13,686::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108579::WARNING::2014-10-03 18:25:13,686::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 18:25:13,687::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 18:25:13,687::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 18:25:23,688::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 18:25:23,688::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108580::WARNING::2014-10-03 18:26:23,688::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 18:26:23,688::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 18:26:23,689::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 18:26:33,690::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 18:26:33,690::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108581::WARNING::2014-10-03 18:27:33,690::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 18:27:33,690::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 18:27:33,690::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 18:27:43,692::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 18:27:43,692::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108582::WARNING::2014-10-03 18:28:43,692::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 18:28:43,692::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 18:28:43,692::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 18:28:53,694::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 18:28:53,694::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108583::WARNING::2014-10-03 18:29:53,693::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 18:29:53,694::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 18:29:53,694::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 18:30:03,695::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 18:30:03,696::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 18:30:03,696::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108584::WARNING::2014-10-03 18:31:03,696::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 18:31:03,696::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 18:31:03,696::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 18:31:11,196::__init__::232::IOProcess::(_processLogs) (92083) Queuing response >Thread-18::DEBUG::2014-10-03 18:31:11,197::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 18:31:11,197::__init__::232::IOProcess::(_processLogs) (92143) Got request for method 'glob' >Thread-18::DEBUG::2014-10-03 18:31:13,697::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 18:31:13,698::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108585::WARNING::2014-10-03 18:32:13,697::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 18:32:13,698::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 18:32:13,698::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 18:32:23,699::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 18:32:23,700::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108586::WARNING::2014-10-03 18:33:23,699::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 18:33:23,700::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 18:33:23,700::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 18:33:33,701::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 18:33:33,701::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108587::WARNING::2014-10-03 18:34:33,701::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 18:34:33,702::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 18:34:33,702::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 18:34:43,703::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 18:34:43,703::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108588::WARNING::2014-10-03 18:35:43,703::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 18:35:43,704::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 18:35:43,704::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 18:35:53,704::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 18:35:53,705::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 18:35:53,705::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108589::WARNING::2014-10-03 18:36:53,705::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 18:36:53,706::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 18:36:53,706::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 18:37:03,707::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 18:37:03,707::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108590::WARNING::2014-10-03 18:38:03,707::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 18:38:03,708::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 18:38:03,708::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 18:38:13,709::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 18:38:13,709::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108591::WARNING::2014-10-03 18:39:13,709::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 18:39:13,710::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 18:39:13,710::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 18:39:23,711::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 18:39:23,711::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108592::WARNING::2014-10-03 18:40:23,711::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 18:40:23,712::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 18:40:23,712::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 18:40:33,713::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 18:40:33,713::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108593::WARNING::2014-10-03 18:41:33,713::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 18:41:33,713::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 18:41:33,714::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 18:41:43,714::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 18:41:43,715::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 18:41:43,715::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108594::WARNING::2014-10-03 18:42:43,715::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 18:42:43,716::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 18:42:43,716::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 18:42:53,717::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 18:42:53,717::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108595::WARNING::2014-10-03 18:43:53,717::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 18:43:53,718::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 18:43:53,718::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 18:44:03,719::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 18:44:03,720::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108596::WARNING::2014-10-03 18:45:03,719::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 18:45:03,720::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 18:45:03,720::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 18:45:13,721::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 18:45:13,721::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108597::WARNING::2014-10-03 18:46:13,721::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 18:46:13,722::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 18:46:13,722::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 18:46:23,723::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 18:46:23,724::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108598::WARNING::2014-10-03 18:47:23,723::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 18:47:23,724::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 18:47:23,724::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 18:47:33,725::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 18:47:33,726::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 18:47:33,726::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108599::WARNING::2014-10-03 18:48:33,725::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 18:48:33,726::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 18:48:33,726::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 18:48:43,728::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 18:48:43,728::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108600::WARNING::2014-10-03 18:49:43,727::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 18:49:43,728::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 18:49:43,728::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 18:49:53,729::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 18:49:53,730::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108601::WARNING::2014-10-03 18:50:53,729::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 18:50:53,730::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 18:50:53,730::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 18:51:03,732::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 18:51:03,732::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108602::WARNING::2014-10-03 18:52:03,731::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 18:52:03,732::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 18:52:03,732::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 18:52:13,733::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 18:52:13,734::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108603::WARNING::2014-10-03 18:53:13,733::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 18:53:13,734::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 18:53:13,734::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 18:53:23,734::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 18:53:23,735::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 18:53:23,735::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108604::WARNING::2014-10-03 18:54:23,735::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 18:54:23,736::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 18:54:23,737::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 18:54:33,738::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 18:54:33,739::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108605::WARNING::2014-10-03 18:55:33,738::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 18:55:33,739::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 18:55:33,739::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 18:55:43,740::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 18:55:43,740::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108606::WARNING::2014-10-03 18:56:43,740::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 18:56:43,741::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 18:56:43,741::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 18:56:53,742::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 18:56:53,742::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108607::WARNING::2014-10-03 18:57:53,742::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 18:57:53,743::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 18:57:53,743::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 18:58:03,744::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 18:58:03,744::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108608::WARNING::2014-10-03 18:59:03,744::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 18:59:03,745::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 18:59:03,745::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 18:59:13,745::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 18:59:13,746::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 18:59:13,746::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108609::WARNING::2014-10-03 19:00:13,746::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 19:00:13,747::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 19:00:13,747::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 19:00:23,748::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 19:00:23,748::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108610::WARNING::2014-10-03 19:01:23,748::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 19:01:23,749::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 19:01:23,749::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 19:01:33,750::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 19:01:33,750::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108611::WARNING::2014-10-03 19:02:33,750::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 19:02:33,750::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 19:02:33,751::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 19:02:43,752::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 19:02:43,752::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108612::WARNING::2014-10-03 19:03:43,752::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 19:03:43,752::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 19:03:43,752::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 19:03:53,753::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 19:03:53,754::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108613::WARNING::2014-10-03 19:04:53,753::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 19:04:53,754::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 19:04:53,754::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 19:05:03,754::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 19:05:03,755::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 19:05:03,755::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108614::WARNING::2014-10-03 19:06:03,755::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 19:06:03,756::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 19:06:03,756::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 19:06:13,757::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 19:06:13,757::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108615::WARNING::2014-10-03 19:07:13,757::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 19:07:13,758::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 19:07:13,758::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 19:07:23,759::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 19:07:23,759::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108616::WARNING::2014-10-03 19:08:23,759::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 19:08:23,759::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 19:08:23,759::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 19:08:33,761::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 19:08:33,761::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108617::WARNING::2014-10-03 19:09:33,761::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 19:09:33,761::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 19:09:33,761::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 19:09:43,762::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 19:09:43,763::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108618::WARNING::2014-10-03 19:10:43,762::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 19:10:43,763::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 19:10:43,763::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 19:10:53,763::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 19:10:53,764::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 19:10:53,764::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108619::WARNING::2014-10-03 19:11:53,764::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 19:11:53,765::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 19:11:53,765::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 19:12:03,766::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 19:12:03,766::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108620::WARNING::2014-10-03 19:13:03,766::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 19:13:03,767::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 19:13:03,767::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 19:13:13,768::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 19:13:13,768::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108621::WARNING::2014-10-03 19:14:13,768::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 19:14:13,768::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 19:14:13,769::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 19:14:23,770::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 19:14:23,770::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108622::WARNING::2014-10-03 19:15:23,770::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 19:15:23,770::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 19:15:23,771::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 19:15:33,772::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 19:15:33,772::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108623::WARNING::2014-10-03 19:16:33,772::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 19:16:33,772::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 19:16:33,772::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 19:16:43,773::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 19:16:43,774::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 19:16:43,774::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108624::WARNING::2014-10-03 19:17:43,774::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 19:17:43,774::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 19:17:43,774::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 19:17:53,776::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 19:17:53,776::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108625::WARNING::2014-10-03 19:18:53,775::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 19:18:53,776::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 19:18:53,776::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 19:19:03,777::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 19:19:03,778::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108626::WARNING::2014-10-03 19:20:03,777::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 19:20:03,778::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 19:20:03,778::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 19:20:13,779::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 19:20:13,780::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108627::WARNING::2014-10-03 19:21:13,779::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 19:21:13,780::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 19:21:13,780::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 19:21:23,781::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 19:21:23,782::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108628::WARNING::2014-10-03 19:22:23,781::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 19:22:23,782::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 19:22:23,782::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 19:22:33,783::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 19:22:33,783::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 19:22:33,784::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108629::WARNING::2014-10-03 19:23:33,783::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 19:23:33,784::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 19:23:33,784::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 19:23:43,785::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 19:23:43,785::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108630::WARNING::2014-10-03 19:24:43,785::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 19:24:43,786::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 19:24:43,786::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 19:24:53,787::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 19:24:53,788::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108631::WARNING::2014-10-03 19:25:53,787::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 19:25:53,788::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 19:25:53,788::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 19:26:03,789::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 19:26:03,789::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108632::WARNING::2014-10-03 19:27:03,789::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 19:27:03,790::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 19:27:03,790::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 19:27:13,791::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 19:27:13,791::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108633::WARNING::2014-10-03 19:28:13,791::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 19:28:13,792::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 19:28:13,792::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 19:28:23,792::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 19:28:23,793::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 19:28:23,793::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108634::WARNING::2014-10-03 19:29:23,793::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 19:29:23,794::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 19:29:23,794::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 19:29:33,795::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 19:29:33,795::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108635::WARNING::2014-10-03 19:30:33,795::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 19:30:33,795::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 19:30:33,796::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 19:30:43,797::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 19:30:43,797::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108636::WARNING::2014-10-03 19:31:43,797::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 19:31:43,797::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 19:31:43,798::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 19:31:53,799::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 19:31:53,799::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108637::WARNING::2014-10-03 19:32:53,799::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 19:32:53,799::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 19:32:53,799::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 19:33:03,801::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 19:33:03,801::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108638::WARNING::2014-10-03 19:34:03,801::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 19:34:03,801::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 19:34:03,801::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 19:34:13,802::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 19:34:13,803::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 19:34:13,803::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108639::WARNING::2014-10-03 19:35:13,803::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 19:35:13,803::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 19:35:13,803::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 19:35:23,804::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 19:35:23,805::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108640::WARNING::2014-10-03 19:36:23,804::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 19:36:23,805::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 19:36:23,805::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 19:36:33,806::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 19:36:33,807::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108641::WARNING::2014-10-03 19:37:33,806::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 19:37:33,807::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 19:37:33,807::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 19:37:43,808::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 19:37:43,808::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108642::WARNING::2014-10-03 19:38:43,808::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 19:38:43,809::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 19:38:43,809::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 19:38:53,810::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 19:38:53,810::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108643::WARNING::2014-10-03 19:39:53,810::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 19:39:53,811::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 19:39:53,811::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 19:40:03,811::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 19:40:03,812::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 19:40:03,812::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108644::WARNING::2014-10-03 19:41:03,812::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 19:41:03,813::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 19:41:03,813::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 19:41:13,814::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 19:41:13,814::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108645::WARNING::2014-10-03 19:42:13,814::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 19:42:13,814::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 19:42:13,815::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 19:42:23,816::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 19:42:23,816::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108646::WARNING::2014-10-03 19:43:23,816::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 19:43:23,816::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 19:43:23,817::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 19:43:33,818::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 19:43:33,818::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108647::WARNING::2014-10-03 19:44:33,818::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 19:44:33,818::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 19:44:33,819::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 19:44:43,820::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 19:44:43,820::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108648::WARNING::2014-10-03 19:45:43,820::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 19:45:43,820::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 19:45:43,821::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 19:45:53,821::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 19:45:53,822::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 19:45:53,822::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108649::WARNING::2014-10-03 19:46:53,822::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 19:46:53,822::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 19:46:53,823::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 19:47:03,824::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 19:47:03,824::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108650::WARNING::2014-10-03 19:48:03,824::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 19:48:03,824::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 19:48:03,825::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 19:48:13,826::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 19:48:13,826::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108651::WARNING::2014-10-03 19:49:13,826::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 19:49:13,826::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 19:49:13,827::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 19:49:23,828::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 19:49:23,828::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108652::WARNING::2014-10-03 19:50:23,828::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 19:50:23,828::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 19:50:23,829::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 19:50:33,830::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 19:50:33,830::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108653::WARNING::2014-10-03 19:51:33,830::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 19:51:33,831::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 19:51:33,831::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 19:51:43,831::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 19:51:43,832::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 19:51:43,833::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-03 19:51:44,476::__init__::232::IOProcess::(_processLogs) (92084) Queuing response >Thread-18::DEBUG::2014-10-03 19:51:44,477::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 19:51:44,477::__init__::232::IOProcess::(_processLogs) (92144) Got request for method 'glob' >Thread-108654::WARNING::2014-10-03 19:52:43,832::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 19:52:43,833::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 19:52:43,833::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 19:52:53,834::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 19:52:53,834::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108655::WARNING::2014-10-03 19:53:53,834::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 19:53:53,835::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 19:53:53,835::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 19:54:03,836::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 19:54:03,836::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108656::WARNING::2014-10-03 19:55:03,836::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 19:55:03,836::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 19:55:03,837::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 19:55:13,838::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 19:55:13,838::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108657::WARNING::2014-10-03 19:56:13,838::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 19:56:13,838::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 19:56:13,838::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 19:56:23,839::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 19:56:23,840::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108658::WARNING::2014-10-03 19:57:23,839::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 19:57:23,840::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 19:57:23,840::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 19:57:33,840::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 19:57:33,841::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 19:57:33,841::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108659::WARNING::2014-10-03 19:58:33,841::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 19:58:33,842::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 19:58:33,842::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 19:58:43,843::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 19:58:43,843::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108660::WARNING::2014-10-03 19:59:43,843::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 19:59:43,844::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 19:59:43,844::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 19:59:53,845::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 19:59:53,845::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108661::WARNING::2014-10-03 20:00:53,845::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 20:00:53,845::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 20:00:53,846::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 20:01:03,847::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 20:01:03,847::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108662::WARNING::2014-10-03 20:02:03,847::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 20:02:03,847::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 20:02:03,847::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 20:02:13,848::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 20:02:13,849::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108663::WARNING::2014-10-03 20:03:13,848::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 20:03:13,849::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 20:03:13,849::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 20:03:23,850::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 20:03:23,851::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 20:03:23,851::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108664::WARNING::2014-10-03 20:04:23,850::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 20:04:23,851::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 20:04:23,851::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 20:04:33,852::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 20:04:33,852::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108665::WARNING::2014-10-03 20:05:33,852::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 20:05:33,853::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 20:05:33,853::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 20:05:43,854::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 20:05:43,854::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108666::WARNING::2014-10-03 20:06:43,854::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 20:06:43,855::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 20:06:43,855::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 20:06:53,856::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 20:06:53,856::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108667::WARNING::2014-10-03 20:07:53,856::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 20:07:53,857::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 20:07:53,857::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 20:08:03,858::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 20:08:03,858::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108668::WARNING::2014-10-03 20:09:03,858::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 20:09:03,858::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 20:09:03,858::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 20:09:13,859::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 20:09:13,860::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 20:09:13,860::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108669::WARNING::2014-10-03 20:10:13,860::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 20:10:13,860::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 20:10:13,860::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 20:10:23,861::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 20:10:23,862::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108670::WARNING::2014-10-03 20:11:23,861::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 20:11:23,862::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 20:11:23,862::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 20:11:33,863::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 20:11:33,863::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108671::WARNING::2014-10-03 20:12:33,863::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 20:12:33,863::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 20:12:33,864::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 20:12:43,865::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 20:12:43,865::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108672::WARNING::2014-10-03 20:13:43,865::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 20:13:43,865::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 20:13:43,865::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 20:13:53,867::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 20:13:53,867::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108673::WARNING::2014-10-03 20:14:53,867::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 20:14:53,867::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 20:14:53,867::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 20:15:03,868::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 20:15:03,869::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 20:15:03,869::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108674::WARNING::2014-10-03 20:16:03,869::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 20:16:03,869::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 20:16:03,869::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 20:16:13,871::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 20:16:13,871::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108675::WARNING::2014-10-03 20:17:13,871::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 20:17:13,871::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 20:17:13,871::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 20:17:23,873::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 20:17:23,873::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108676::WARNING::2014-10-03 20:18:23,872::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 20:18:23,873::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 20:18:23,873::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 20:18:33,875::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 20:18:33,875::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108677::WARNING::2014-10-03 20:19:33,874::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 20:19:33,875::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 20:19:33,875::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 20:19:43,877::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 20:19:43,877::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108678::WARNING::2014-10-03 20:20:43,876::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 20:20:43,877::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 20:20:43,877::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 20:20:53,878::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 20:20:53,879::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 20:20:53,879::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108679::WARNING::2014-10-03 20:21:53,879::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 20:21:53,879::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 20:21:53,879::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 20:22:03,881::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 20:22:03,881::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108680::WARNING::2014-10-03 20:23:03,881::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 20:23:03,881::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 20:23:03,881::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 20:23:13,883::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 20:23:13,883::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108681::WARNING::2014-10-03 20:24:13,882::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 20:24:13,883::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 20:24:13,883::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 20:24:23,884::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 20:24:23,885::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108682::WARNING::2014-10-03 20:25:23,884::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 20:25:23,885::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 20:25:23,885::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 20:25:33,886::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 20:25:33,886::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108683::WARNING::2014-10-03 20:26:33,886::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 20:26:33,887::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 20:26:33,887::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 20:26:43,887::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 20:26:43,888::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 20:26:43,888::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108684::WARNING::2014-10-03 20:27:43,888::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 20:27:43,889::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 20:27:43,889::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 20:27:53,890::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 20:27:53,890::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108685::WARNING::2014-10-03 20:28:53,890::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 20:28:53,891::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 20:28:53,891::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 20:29:03,893::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 20:29:03,894::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108686::WARNING::2014-10-03 20:30:03,893::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 20:30:03,893::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 20:30:03,894::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 20:30:13,895::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 20:30:13,895::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108687::WARNING::2014-10-03 20:31:13,895::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 20:31:13,895::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 20:31:13,895::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 20:31:23,897::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 20:31:23,897::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-03 20:32:01,116::__init__::232::IOProcess::(_processLogs) (92085) Queuing response >Thread-18::DEBUG::2014-10-03 20:32:01,117::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 20:32:01,117::__init__::232::IOProcess::(_processLogs) (92145) Got request for method 'glob' >Thread-108688::WARNING::2014-10-03 20:32:23,897::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 20:32:23,897::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 20:32:23,897::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 20:32:33,898::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 20:32:33,899::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 20:32:33,899::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108689::WARNING::2014-10-03 20:33:33,899::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 20:33:33,899::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 20:33:33,899::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 20:33:43,901::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 20:33:43,901::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108690::WARNING::2014-10-03 20:34:43,901::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 20:34:43,901::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 20:34:43,901::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 20:34:53,902::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 20:34:53,903::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108691::WARNING::2014-10-03 20:35:53,902::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 20:35:53,903::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 20:35:53,903::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 20:36:03,904::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 20:36:03,904::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108692::WARNING::2014-10-03 20:37:03,904::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 20:37:03,905::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 20:37:03,905::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 20:37:13,906::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 20:37:13,906::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108693::WARNING::2014-10-03 20:38:13,906::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 20:38:13,907::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 20:38:13,907::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 20:38:23,907::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 20:38:23,908::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 20:38:23,908::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108694::WARNING::2014-10-03 20:39:23,908::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 20:39:23,909::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 20:39:23,909::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 20:39:33,910::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 20:39:33,910::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108695::WARNING::2014-10-03 20:40:33,910::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 20:40:33,910::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 20:40:33,911::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 20:40:43,912::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 20:40:43,912::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108696::WARNING::2014-10-03 20:41:43,912::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 20:41:43,912::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 20:41:43,913::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 20:41:53,914::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 20:41:53,914::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-03 20:42:05,277::__init__::232::IOProcess::(_processLogs) (92086) Queuing response >Thread-18::DEBUG::2014-10-03 20:42:05,277::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 20:42:05,277::__init__::232::IOProcess::(_processLogs) (92146) Got request for method 'glob' >Thread-108697::WARNING::2014-10-03 20:42:53,914::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 20:42:53,914::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 20:42:53,914::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 20:43:03,916::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 20:43:03,916::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108698::WARNING::2014-10-03 20:44:03,916::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 20:44:03,916::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 20:44:03,916::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 20:44:13,917::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 20:44:13,918::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 20:44:13,918::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108699::WARNING::2014-10-03 20:45:13,918::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 20:45:13,918::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 20:45:13,918::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 20:45:23,920::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 20:45:23,920::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108700::WARNING::2014-10-03 20:46:23,920::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 20:46:23,920::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 20:46:23,920::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 20:46:33,922::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 20:46:33,922::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108701::WARNING::2014-10-03 20:47:33,921::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 20:47:33,922::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 20:47:33,922::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 20:47:43,923::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 20:47:43,924::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108702::WARNING::2014-10-03 20:48:43,923::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 20:48:43,924::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 20:48:43,924::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 20:48:53,925::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 20:48:53,926::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108703::WARNING::2014-10-03 20:49:53,925::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 20:49:53,926::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 20:49:53,926::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 20:50:03,927::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 20:50:03,928::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 20:50:03,928::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108704::WARNING::2014-10-03 20:51:03,928::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 20:51:03,928::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 20:51:03,928::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 20:51:13,930::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 20:51:13,930::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-03 20:52:09,436::__init__::232::IOProcess::(_processLogs) (92087) Queuing response >Thread-18::DEBUG::2014-10-03 20:52:09,437::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 20:52:09,437::__init__::232::IOProcess::(_processLogs) (92147) Got request for method 'glob' >Thread-108705::WARNING::2014-10-03 20:52:13,930::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 20:52:13,930::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 20:52:13,930::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 20:52:23,932::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 20:52:23,932::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108706::WARNING::2014-10-03 20:53:23,932::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 20:53:23,932::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 20:53:23,932::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 20:53:33,933::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 20:53:33,934::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108707::WARNING::2014-10-03 20:54:33,933::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 20:54:33,934::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 20:54:33,934::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 20:54:43,935::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 20:54:43,935::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108708::WARNING::2014-10-03 20:55:43,935::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 20:55:43,936::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 20:55:43,936::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 20:55:53,936::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 20:55:53,937::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 20:55:53,937::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108709::WARNING::2014-10-03 20:56:53,937::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 20:56:53,937::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 20:56:53,938::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 20:57:03,939::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 20:57:03,939::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108710::WARNING::2014-10-03 20:58:03,939::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 20:58:03,939::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 20:58:03,939::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 20:58:13,941::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 20:58:13,941::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108711::WARNING::2014-10-03 20:59:13,941::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 20:59:13,941::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 20:59:13,941::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 20:59:23,942::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 20:59:23,943::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108712::WARNING::2014-10-03 21:00:23,942::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 21:00:23,943::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 21:00:23,943::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 21:00:33,944::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 21:00:33,944::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108713::WARNING::2014-10-03 21:01:33,944::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 21:01:33,945::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 21:01:33,945::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 21:01:43,945::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 21:01:43,946::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 21:01:43,946::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108714::WARNING::2014-10-03 21:02:43,946::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 21:02:43,946::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 21:02:43,947::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 21:02:53,948::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 21:02:53,948::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108715::WARNING::2014-10-03 21:03:53,948::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 21:03:53,948::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 21:03:53,948::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 21:04:03,950::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 21:04:03,950::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108716::WARNING::2014-10-03 21:05:03,949::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 21:05:03,950::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 21:05:03,950::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 21:05:13,951::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 21:05:13,952::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108717::WARNING::2014-10-03 21:06:13,951::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 21:06:13,952::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 21:06:13,952::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 21:06:23,953::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 21:06:23,953::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108718::WARNING::2014-10-03 21:07:23,953::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 21:07:23,953::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 21:07:23,954::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 21:07:33,954::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 21:07:33,955::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 21:07:33,955::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108719::WARNING::2014-10-03 21:08:33,955::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 21:08:33,955::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 21:08:33,956::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 21:08:43,957::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 21:08:43,957::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108720::WARNING::2014-10-03 21:09:43,957::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 21:09:43,957::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 21:09:43,957::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 21:09:53,959::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 21:09:53,959::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108721::WARNING::2014-10-03 21:10:53,959::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 21:10:53,959::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 21:10:53,959::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 21:11:03,960::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 21:11:03,961::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108722::WARNING::2014-10-03 21:12:03,960::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 21:12:03,961::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 21:12:03,961::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 21:12:13,962::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 21:12:13,962::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108723::WARNING::2014-10-03 21:13:13,962::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 21:13:13,963::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 21:13:13,963::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 21:13:23,963::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 21:13:23,964::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 21:13:23,964::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108724::WARNING::2014-10-03 21:14:23,964::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 21:14:23,965::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 21:14:23,965::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 21:14:33,966::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 21:14:33,966::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108725::WARNING::2014-10-03 21:15:33,966::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 21:15:33,967::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 21:15:33,967::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 21:15:43,968::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 21:15:43,968::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108726::WARNING::2014-10-03 21:16:43,968::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 21:16:43,968::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 21:16:43,969::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 21:16:53,970::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 21:16:53,970::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108727::WARNING::2014-10-03 21:17:53,970::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 21:17:53,970::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 21:17:53,971::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 21:18:03,972::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 21:18:03,972::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108728::WARNING::2014-10-03 21:19:03,972::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 21:19:03,972::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 21:19:03,973::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 21:19:13,973::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 21:19:13,974::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 21:19:13,974::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108729::WARNING::2014-10-03 21:20:13,974::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 21:20:13,975::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 21:20:13,975::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 21:20:23,976::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 21:20:23,976::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108730::WARNING::2014-10-03 21:21:23,976::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 21:21:23,977::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 21:21:23,977::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 21:21:33,978::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 21:21:33,978::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108731::WARNING::2014-10-03 21:22:33,978::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 21:22:33,979::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 21:22:33,979::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 21:22:43,980::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 21:22:43,980::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108732::WARNING::2014-10-03 21:23:43,980::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 21:23:43,980::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 21:23:43,981::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 21:23:53,982::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 21:23:53,982::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108733::WARNING::2014-10-03 21:24:53,982::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 21:24:53,982::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 21:24:53,982::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 21:25:03,983::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 21:25:03,984::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 21:25:03,984::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108734::WARNING::2014-10-03 21:26:03,984::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 21:26:03,984::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 21:26:03,984::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 21:26:13,986::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 21:26:13,986::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108735::WARNING::2014-10-03 21:27:13,986::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 21:27:13,986::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 21:27:13,986::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 21:27:23,988::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 21:27:23,988::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108736::WARNING::2014-10-03 21:28:23,988::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 21:28:23,988::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 21:28:23,988::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 21:28:33,989::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 21:28:33,990::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108737::WARNING::2014-10-03 21:29:33,989::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 21:29:33,990::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 21:29:33,990::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 21:29:43,991::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 21:29:43,991::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108738::WARNING::2014-10-03 21:30:43,991::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 21:30:43,992::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 21:30:43,992::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 21:30:53,992::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 21:30:53,993::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 21:30:53,993::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108739::WARNING::2014-10-03 21:31:53,993::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 21:31:53,994::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 21:31:53,994::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 21:32:03,995::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 21:32:03,995::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-03 21:32:26,076::__init__::232::IOProcess::(_processLogs) (92088) Queuing response >Thread-18::DEBUG::2014-10-03 21:32:26,077::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 21:32:26,077::__init__::232::IOProcess::(_processLogs) (92148) Got request for method 'glob' >Thread-108740::WARNING::2014-10-03 21:33:03,995::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 21:33:03,996::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 21:33:03,996::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 21:33:13,997::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 21:33:13,997::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108741::WARNING::2014-10-03 21:34:13,997::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 21:34:13,997::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 21:34:13,998::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 21:34:23,999::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 21:34:23,999::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108742::WARNING::2014-10-03 21:35:23,999::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 21:35:23,999::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 21:35:24,000::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 21:35:34,001::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 21:35:34,001::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108743::WARNING::2014-10-03 21:36:34,001::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 21:36:34,001::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 21:36:34,001::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 21:36:44,002::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 21:36:44,003::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 21:36:44,003::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108744::WARNING::2014-10-03 21:37:44,003::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 21:37:44,003::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 21:37:44,003::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 21:37:54,005::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 21:37:54,005::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108745::WARNING::2014-10-03 21:38:54,004::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 21:38:54,005::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 21:38:54,005::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 21:39:04,006::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 21:39:04,007::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108746::WARNING::2014-10-03 21:40:04,006::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 21:40:04,007::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 21:40:04,007::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 21:40:14,008::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 21:40:14,008::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108747::WARNING::2014-10-03 21:41:14,008::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 21:41:14,009::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 21:41:14,009::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 21:41:24,010::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 21:41:24,010::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108748::WARNING::2014-10-03 21:42:24,010::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 21:42:24,011::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 21:42:24,011::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 21:42:30,236::__init__::232::IOProcess::(_processLogs) (92089) Queuing response >Thread-18::DEBUG::2014-10-03 21:42:30,237::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 21:42:30,237::__init__::232::IOProcess::(_processLogs) (92149) Got request for method 'glob' >Thread-48::DEBUG::2014-10-03 21:42:34,011::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 21:42:34,012::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 21:42:34,012::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108749::WARNING::2014-10-03 21:43:34,012::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 21:43:34,013::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 21:43:34,013::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 21:43:44,014::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 21:43:44,014::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108750::WARNING::2014-10-03 21:44:44,014::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 21:44:44,014::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 21:44:44,015::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 21:44:54,016::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 21:44:54,016::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108751::WARNING::2014-10-03 21:45:54,016::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 21:45:54,016::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 21:45:54,017::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 21:46:04,018::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 21:46:04,018::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108752::WARNING::2014-10-03 21:47:04,018::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 21:47:04,018::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 21:47:04,018::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 21:47:14,020::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 21:47:14,020::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108753::WARNING::2014-10-03 21:48:14,020::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 21:48:14,020::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 21:48:14,020::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 21:48:24,021::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 21:48:24,022::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 21:48:24,022::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108754::WARNING::2014-10-03 21:49:24,022::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 21:49:24,022::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 21:49:24,022::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 21:49:34,024::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 21:49:34,024::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108755::WARNING::2014-10-03 21:50:34,024::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 21:50:34,024::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 21:50:34,024::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 21:50:44,026::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 21:50:44,026::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108756::WARNING::2014-10-03 21:51:44,026::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 21:51:44,026::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 21:51:44,026::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 21:51:54,028::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 21:51:54,028::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108757::WARNING::2014-10-03 21:52:54,028::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 21:52:54,028::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 21:52:54,028::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 21:53:04,029::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 21:53:04,030::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108758::WARNING::2014-10-03 21:54:04,029::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 21:54:04,030::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 21:54:04,030::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 21:54:14,030::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 21:54:14,031::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 21:54:14,031::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108759::WARNING::2014-10-03 21:55:14,031::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 21:55:14,032::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 21:55:14,032::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 21:55:24,033::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 21:55:24,033::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108760::WARNING::2014-10-03 21:56:24,033::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 21:56:24,033::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 21:56:24,034::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 21:56:34,035::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 21:56:34,035::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108761::WARNING::2014-10-03 21:57:34,035::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 21:57:34,035::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 21:57:34,035::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 21:57:44,036::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 21:57:44,037::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108762::WARNING::2014-10-03 21:58:44,036::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 21:58:44,037::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 21:58:44,037::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 21:58:54,038::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 21:58:54,038::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108763::WARNING::2014-10-03 21:59:54,038::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 21:59:54,039::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 21:59:54,039::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 22:00:04,039::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 22:00:04,040::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 22:00:04,040::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108764::WARNING::2014-10-03 22:01:04,040::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 22:01:04,040::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 22:01:04,041::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 22:01:14,042::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 22:01:14,042::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108765::WARNING::2014-10-03 22:02:14,042::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 22:02:14,042::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 22:02:14,042::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 22:02:24,044::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 22:02:24,044::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108766::WARNING::2014-10-03 22:03:24,043::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 22:03:24,044::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 22:03:24,044::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 22:03:34,045::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 22:03:34,045::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108767::WARNING::2014-10-03 22:04:34,045::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 22:04:34,046::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 22:04:34,046::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 22:04:44,047::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 22:04:44,047::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108768::WARNING::2014-10-03 22:05:44,047::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 22:05:44,048::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 22:05:44,048::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 22:05:54,049::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 22:05:54,050::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 22:05:54,050::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108769::WARNING::2014-10-03 22:06:54,049::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 22:06:54,050::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 22:06:54,050::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 22:07:04,051::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 22:07:04,052::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108770::WARNING::2014-10-03 22:08:04,051::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 22:08:04,052::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 22:08:04,052::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 22:08:14,053::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 22:08:14,054::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108771::WARNING::2014-10-03 22:09:14,053::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 22:09:14,054::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 22:09:14,054::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 22:09:24,055::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 22:09:24,055::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108772::WARNING::2014-10-03 22:10:24,055::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 22:10:24,056::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 22:10:24,056::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 22:10:34,057::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 22:10:34,057::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108773::WARNING::2014-10-03 22:11:34,057::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 22:11:34,057::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 22:11:34,058::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 22:11:44,058::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 22:11:44,059::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 22:11:44,059::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108774::WARNING::2014-10-03 22:12:44,059::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 22:12:44,060::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 22:12:44,060::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 22:12:54,061::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 22:12:54,061::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108775::WARNING::2014-10-03 22:13:54,061::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 22:13:54,061::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 22:13:54,062::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 22:14:04,063::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 22:14:04,063::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108776::WARNING::2014-10-03 22:15:04,063::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 22:15:04,063::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 22:15:04,063::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 22:15:14,065::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 22:15:14,065::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108777::WARNING::2014-10-03 22:16:14,065::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 22:16:14,065::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 22:16:14,065::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 22:16:24,067::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 22:16:24,067::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108778::WARNING::2014-10-03 22:17:24,066::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 22:17:24,067::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 22:17:24,067::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 22:17:34,068::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 22:17:34,069::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 22:17:34,069::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108779::WARNING::2014-10-03 22:18:34,069::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 22:18:34,069::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 22:18:34,069::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 22:18:44,071::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 22:18:44,071::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108780::WARNING::2014-10-03 22:19:44,071::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 22:19:44,071::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 22:19:44,071::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 22:19:54,073::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 22:19:54,073::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108781::WARNING::2014-10-03 22:20:54,073::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 22:20:54,073::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 22:20:54,074::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 22:21:04,075::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 22:21:04,075::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108782::WARNING::2014-10-03 22:22:04,075::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 22:22:04,075::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 22:22:04,075::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 22:22:14,077::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 22:22:14,077::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108783::WARNING::2014-10-03 22:23:14,077::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 22:23:14,077::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 22:23:14,077::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 22:23:24,078::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 22:23:24,079::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 22:23:24,079::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108784::WARNING::2014-10-03 22:24:24,078::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 22:24:24,079::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 22:24:24,079::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 22:24:34,080::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 22:24:34,081::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108785::WARNING::2014-10-03 22:25:34,080::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 22:25:34,081::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 22:25:34,081::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 22:25:44,082::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 22:25:44,082::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108786::WARNING::2014-10-03 22:26:44,082::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 22:26:44,083::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 22:26:44,083::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 22:26:54,084::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 22:26:54,084::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108787::WARNING::2014-10-03 22:27:54,084::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 22:27:54,085::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 22:27:54,085::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 22:28:04,086::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 22:28:04,086::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108788::WARNING::2014-10-03 22:29:04,086::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 22:29:04,087::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 22:29:04,087::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 22:29:14,087::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 22:29:14,088::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 22:29:14,088::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108789::WARNING::2014-10-03 22:30:14,088::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 22:30:14,089::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 22:30:14,089::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 22:30:24,090::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 22:30:24,090::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108790::WARNING::2014-10-03 22:31:24,090::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 22:31:24,090::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 22:31:24,091::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 22:31:34,092::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 22:31:34,092::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108791::WARNING::2014-10-03 22:32:34,092::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 22:32:34,092::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 22:32:34,093::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 22:32:44,094::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 22:32:44,094::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108792::WARNING::2014-10-03 22:33:44,094::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 22:33:44,094::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 22:33:44,094::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 22:33:54,096::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 22:33:54,096::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108793::WARNING::2014-10-03 22:34:54,096::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 22:34:54,096::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 22:34:54,096::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 22:35:04,097::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 22:35:04,098::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 22:35:04,098::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108794::WARNING::2014-10-03 22:36:04,098::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 22:36:04,098::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 22:36:04,098::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 22:36:14,100::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 22:36:14,100::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108795::WARNING::2014-10-03 22:37:14,099::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 22:37:14,100::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 22:37:14,100::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 22:37:24,101::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 22:37:24,102::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108796::WARNING::2014-10-03 22:38:24,101::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 22:38:24,102::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 22:38:24,102::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 22:38:34,103::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 22:38:34,103::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108797::WARNING::2014-10-03 22:39:34,103::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 22:39:34,104::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 22:39:34,104::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 22:39:44,105::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 22:39:44,105::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108798::WARNING::2014-10-03 22:40:44,105::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 22:40:44,106::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 22:40:44,106::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 22:40:54,106::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 22:40:54,107::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 22:40:54,107::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108799::WARNING::2014-10-03 22:41:54,107::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 22:41:54,108::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 22:41:54,108::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 22:42:04,109::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 22:42:04,109::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108800::WARNING::2014-10-03 22:43:04,109::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 22:43:04,110::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 22:43:04,110::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 22:43:14,111::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 22:43:14,111::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108801::WARNING::2014-10-03 22:44:14,111::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 22:44:14,112::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 22:44:14,112::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 22:44:24,113::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 22:44:24,113::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108802::WARNING::2014-10-03 22:45:24,113::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 22:45:24,114::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 22:45:24,114::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 22:45:34,115::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 22:45:34,115::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108803::WARNING::2014-10-03 22:46:34,115::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 22:46:34,116::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 22:46:34,116::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 22:46:44,116::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 22:46:44,117::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 22:46:44,117::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108804::WARNING::2014-10-03 22:47:44,117::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 22:47:44,118::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 22:47:44,118::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 22:47:54,119::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 22:47:54,119::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108805::WARNING::2014-10-03 22:48:54,119::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 22:48:54,120::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 22:48:54,120::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 22:49:04,121::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 22:49:04,121::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108806::WARNING::2014-10-03 22:50:04,121::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 22:50:04,122::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 22:50:04,122::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 22:50:14,123::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 22:50:14,123::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108807::WARNING::2014-10-03 22:51:14,123::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 22:51:14,124::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 22:51:14,124::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 22:51:24,125::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 22:51:24,125::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108808::WARNING::2014-10-03 22:52:24,125::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 22:52:24,126::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 22:52:24,126::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 22:52:34,126::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 22:52:34,127::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 22:52:34,127::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-03 22:52:59,357::__init__::232::IOProcess::(_processLogs) (92090) Queuing response >Thread-18::DEBUG::2014-10-03 22:52:59,357::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 22:52:59,357::__init__::232::IOProcess::(_processLogs) (92150) Got request for method 'glob' >Thread-108809::WARNING::2014-10-03 22:53:34,127::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 22:53:34,128::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 22:53:34,128::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 22:53:44,129::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 22:53:44,129::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108810::WARNING::2014-10-03 22:54:44,129::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 22:54:44,129::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 22:54:44,129::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 22:54:54,131::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 22:54:54,131::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108811::WARNING::2014-10-03 22:55:54,131::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 22:55:54,131::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 22:55:54,131::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 22:56:04,132::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 22:56:04,133::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108812::WARNING::2014-10-03 22:57:04,132::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 22:57:04,133::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 22:57:04,133::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 22:57:14,135::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 22:57:14,135::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108813::WARNING::2014-10-03 22:58:14,134::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 22:58:14,135::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 22:58:14,135::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 22:58:24,135::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 22:58:24,136::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 22:58:24,137::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108814::WARNING::2014-10-03 22:59:24,136::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 22:59:24,137::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 22:59:24,137::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 22:59:34,139::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 22:59:34,140::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108815::WARNING::2014-10-03 23:00:34,139::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 23:00:34,140::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 23:00:34,140::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 23:00:44,141::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 23:00:44,141::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108816::WARNING::2014-10-03 23:01:44,141::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 23:01:44,142::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 23:01:44,142::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 23:01:54,143::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 23:01:54,143::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108817::WARNING::2014-10-03 23:02:54,143::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 23:02:54,144::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 23:02:54,144::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 23:03:03,516::__init__::232::IOProcess::(_processLogs) (92091) Queuing response >Thread-18::DEBUG::2014-10-03 23:03:03,517::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 23:03:03,517::__init__::232::IOProcess::(_processLogs) (92151) Got request for method 'glob' >Thread-18::DEBUG::2014-10-03 23:03:03,517::__init__::232::IOProcess::(_processLogs) (92092) Queuing response >Thread-18::DEBUG::2014-10-03 23:03:03,517::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 23:03:03,517::__init__::232::IOProcess::(_processLogs) (92152) Got request for method 'glob' >Thread-18::DEBUG::2014-10-03 23:03:04,145::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 23:03:04,145::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108818::WARNING::2014-10-03 23:04:04,145::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 23:04:04,145::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 23:04:04,146::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 23:04:14,146::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 23:04:14,147::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 23:04:14,147::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108819::WARNING::2014-10-03 23:05:14,147::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 23:05:14,148::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 23:05:14,148::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 23:05:24,149::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 23:05:24,149::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108820::WARNING::2014-10-03 23:06:24,149::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 23:06:24,150::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 23:06:24,150::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 23:06:34,151::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 23:06:34,151::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108821::WARNING::2014-10-03 23:07:34,151::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 23:07:34,152::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 23:07:34,152::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 23:07:44,153::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 23:07:44,153::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108822::WARNING::2014-10-03 23:08:44,153::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 23:08:44,153::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 23:08:44,154::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 23:08:54,155::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 23:08:54,155::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108823::WARNING::2014-10-03 23:09:54,155::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 23:09:54,155::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 23:09:54,156::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 23:10:04,156::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 23:10:04,157::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 23:10:04,157::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108824::WARNING::2014-10-03 23:11:04,157::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 23:11:04,157::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 23:11:04,158::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 23:11:14,159::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 23:11:14,159::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108825::WARNING::2014-10-03 23:12:14,159::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 23:12:14,159::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 23:12:14,159::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 23:12:24,161::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 23:12:24,161::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-03 23:13:07,676::__init__::232::IOProcess::(_processLogs) (92093) Queuing response >Thread-18::DEBUG::2014-10-03 23:13:07,677::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 23:13:07,677::__init__::232::IOProcess::(_processLogs) (92153) Got request for method 'glob' >Thread-18::DEBUG::2014-10-03 23:13:07,677::__init__::232::IOProcess::(_processLogs) (92094) Queuing response >Thread-18::DEBUG::2014-10-03 23:13:07,677::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 23:13:07,677::__init__::232::IOProcess::(_processLogs) (92154) Got request for method 'glob' >Thread-108826::WARNING::2014-10-03 23:13:24,161::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 23:13:24,161::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 23:13:24,161::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 23:13:34,163::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 23:13:34,163::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108827::WARNING::2014-10-03 23:14:34,163::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 23:14:34,163::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 23:14:34,163::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 23:14:44,165::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 23:14:44,165::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108828::WARNING::2014-10-03 23:15:44,165::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 23:15:44,165::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 23:15:44,165::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 23:15:54,166::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 23:15:54,167::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 23:15:54,167::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108829::WARNING::2014-10-03 23:16:54,167::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 23:16:54,167::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 23:16:54,167::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 23:17:04,169::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 23:17:04,169::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108830::WARNING::2014-10-03 23:18:04,169::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 23:18:04,169::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 23:18:04,169::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 23:18:14,171::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 23:18:14,171::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108831::WARNING::2014-10-03 23:19:14,170::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 23:19:14,171::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 23:19:14,171::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 23:19:24,172::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 23:19:24,173::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108832::WARNING::2014-10-03 23:20:24,172::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 23:20:24,173::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 23:20:24,173::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 23:20:34,174::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 23:20:34,175::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108833::WARNING::2014-10-03 23:21:34,174::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 23:21:34,175::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 23:21:34,175::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 23:21:44,176::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 23:21:44,177::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 23:21:44,177::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108834::WARNING::2014-10-03 23:22:44,176::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 23:22:44,177::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 23:22:44,177::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 23:22:54,178::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 23:22:54,179::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-03 23:23:11,836::__init__::232::IOProcess::(_processLogs) (92095) Queuing response >Thread-18::DEBUG::2014-10-03 23:23:11,837::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 23:23:11,837::__init__::232::IOProcess::(_processLogs) (92155) Got request for method 'glob' >Thread-18::DEBUG::2014-10-03 23:23:11,837::__init__::232::IOProcess::(_processLogs) (92096) Queuing response >Thread-18::DEBUG::2014-10-03 23:23:11,837::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 23:23:11,837::__init__::232::IOProcess::(_processLogs) (92156) Got request for method 'glob' >Thread-108835::WARNING::2014-10-03 23:23:54,178::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 23:23:54,179::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 23:23:54,179::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 23:24:04,180::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 23:24:04,180::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108836::WARNING::2014-10-03 23:25:04,180::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 23:25:04,181::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 23:25:04,181::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 23:25:14,182::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 23:25:14,182::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108837::WARNING::2014-10-03 23:26:14,182::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 23:26:14,183::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 23:26:14,183::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 23:26:24,184::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 23:26:24,184::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108838::WARNING::2014-10-03 23:27:24,184::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 23:27:24,184::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 23:27:24,185::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 23:27:34,185::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 23:27:34,186::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 23:27:34,186::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108839::WARNING::2014-10-03 23:28:34,186::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 23:28:34,187::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 23:28:34,187::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 23:28:44,188::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 23:28:44,188::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108840::WARNING::2014-10-03 23:29:44,188::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 23:29:44,189::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 23:29:44,189::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 23:29:54,190::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 23:29:54,190::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108841::WARNING::2014-10-03 23:30:54,190::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 23:30:54,191::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 23:30:54,191::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 23:31:04,192::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 23:31:04,192::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108842::WARNING::2014-10-03 23:32:04,192::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 23:32:04,193::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 23:32:04,193::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 23:32:14,194::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 23:32:14,194::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108843::WARNING::2014-10-03 23:33:14,194::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 23:33:14,194::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 23:33:14,195::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 23:33:15,996::__init__::232::IOProcess::(_processLogs) (92097) Queuing response >Thread-18::DEBUG::2014-10-03 23:33:15,996::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 23:33:15,997::__init__::232::IOProcess::(_processLogs) (92157) Got request for method 'glob' >Thread-18::DEBUG::2014-10-03 23:33:15,997::__init__::232::IOProcess::(_processLogs) (92098) Queuing response >Thread-18::DEBUG::2014-10-03 23:33:15,997::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 23:33:15,997::__init__::232::IOProcess::(_processLogs) (92158) Got request for method 'glob' >Thread-48::DEBUG::2014-10-03 23:33:24,195::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 23:33:24,196::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 23:33:24,196::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108844::WARNING::2014-10-03 23:34:24,196::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 23:34:24,197::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 23:34:24,197::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 23:34:34,198::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 23:34:34,198::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108845::WARNING::2014-10-03 23:35:34,198::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 23:35:34,198::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 23:35:34,199::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 23:35:44,200::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 23:35:44,200::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108846::WARNING::2014-10-03 23:36:44,200::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 23:36:44,200::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 23:36:44,201::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 23:36:54,202::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 23:36:54,202::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108847::WARNING::2014-10-03 23:37:54,202::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 23:37:54,202::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 23:37:54,203::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 23:38:04,204::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 23:38:04,204::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108848::WARNING::2014-10-03 23:39:04,204::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 23:39:04,204::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 23:39:04,204::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 23:39:14,205::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 23:39:14,206::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 23:39:14,206::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108849::WARNING::2014-10-03 23:40:14,206::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 23:40:14,206::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 23:40:14,206::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 23:40:24,208::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 23:40:24,208::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108850::WARNING::2014-10-03 23:41:24,208::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 23:41:24,208::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 23:41:24,208::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 23:41:34,210::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 23:41:34,210::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108851::WARNING::2014-10-03 23:42:34,209::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 23:42:34,210::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 23:42:34,210::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 23:42:44,211::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 23:42:44,212::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-03 23:43:20,156::__init__::232::IOProcess::(_processLogs) (92099) Queuing response >Thread-18::DEBUG::2014-10-03 23:43:20,157::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 23:43:20,157::__init__::232::IOProcess::(_processLogs) (92159) Got request for method 'glob' >Thread-18::DEBUG::2014-10-03 23:43:20,158::__init__::232::IOProcess::(_processLogs) (92100) Queuing response >Thread-18::DEBUG::2014-10-03 23:43:20,159::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 23:43:20,159::__init__::232::IOProcess::(_processLogs) (92160) Got request for method 'glob' >Thread-108852::WARNING::2014-10-03 23:43:44,211::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 23:43:44,212::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 23:43:44,212::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 23:43:54,213::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 23:43:54,214::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108853::WARNING::2014-10-03 23:44:54,213::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 23:44:54,214::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 23:44:54,214::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 23:45:04,215::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 23:45:04,216::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 23:45:04,216::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108854::WARNING::2014-10-03 23:46:04,215::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 23:46:04,216::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 23:46:04,216::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 23:46:14,217::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 23:46:14,218::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108855::WARNING::2014-10-03 23:47:14,217::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 23:47:14,218::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 23:47:14,218::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 23:47:24,219::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 23:47:24,219::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108856::WARNING::2014-10-03 23:48:24,219::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 23:48:24,220::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 23:48:24,220::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 23:48:34,221::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 23:48:34,221::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108857::WARNING::2014-10-03 23:49:34,221::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 23:49:34,222::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 23:49:34,222::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 23:49:44,223::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 23:49:44,223::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108858::WARNING::2014-10-03 23:50:44,223::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 23:50:44,223::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 23:50:44,224::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 23:50:54,224::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 23:50:54,225::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 23:50:54,225::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108859::WARNING::2014-10-03 23:51:54,225::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 23:51:54,226::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 23:51:54,226::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 23:52:04,227::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 23:52:04,227::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108860::WARNING::2014-10-03 23:53:04,227::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 23:53:04,227::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 23:53:04,228::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 23:53:14,229::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 23:53:14,229::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-03 23:53:26,364::__init__::232::IOProcess::(_processLogs) (92101) Queuing response >Thread-18::DEBUG::2014-10-03 23:53:26,365::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 23:53:26,365::__init__::232::IOProcess::(_processLogs) (92161) Got request for method 'glob' >Thread-18::DEBUG::2014-10-03 23:53:26,365::__init__::232::IOProcess::(_processLogs) (92102) Queuing response >Thread-18::DEBUG::2014-10-03 23:53:26,365::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-03 23:53:26,365::__init__::232::IOProcess::(_processLogs) (92162) Got request for method 'glob' >Thread-108861::WARNING::2014-10-03 23:54:14,229::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 23:54:14,229::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 23:54:14,230::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 23:54:24,231::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 23:54:24,231::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108862::WARNING::2014-10-03 23:55:24,231::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 23:55:24,231::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 23:55:24,231::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 23:55:34,233::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 23:55:34,233::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108863::WARNING::2014-10-03 23:56:34,232::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 23:56:34,233::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 23:56:34,233::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-03 23:56:44,234::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-03 23:56:44,235::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 23:56:44,235::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108864::WARNING::2014-10-03 23:57:44,235::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 23:57:44,235::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 23:57:44,235::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 23:57:54,237::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 23:57:54,237::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108865::WARNING::2014-10-03 23:58:54,236::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-03 23:58:54,237::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-03 23:58:54,237::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-03 23:59:04,238::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-03 23:59:04,239::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108866::WARNING::2014-10-04 00:00:04,238::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 00:00:04,239::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 00:00:04,239::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 00:00:14,240::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 00:00:14,240::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108867::WARNING::2014-10-04 00:01:14,240::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 00:01:14,241::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 00:01:14,241::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 00:01:24,243::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 00:01:24,243::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108868::WARNING::2014-10-04 00:02:24,243::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 00:02:24,243::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 00:02:24,244::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 00:02:34,244::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 00:02:34,245::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 00:02:34,245::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-04 00:03:32,572::__init__::232::IOProcess::(_processLogs) (92103) Queuing response >Thread-18::DEBUG::2014-10-04 00:03:32,573::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 00:03:32,573::__init__::232::IOProcess::(_processLogs) (92163) Got request for method 'glob' >Thread-18::DEBUG::2014-10-04 00:03:32,573::__init__::232::IOProcess::(_processLogs) (92104) Queuing response >Thread-18::DEBUG::2014-10-04 00:03:32,573::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 00:03:32,573::__init__::232::IOProcess::(_processLogs) (92164) Got request for method 'glob' >Thread-108869::WARNING::2014-10-04 00:03:34,245::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 00:03:34,245::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 00:03:34,246::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 00:03:44,247::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 00:03:44,247::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108870::WARNING::2014-10-04 00:04:44,247::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 00:04:44,247::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 00:04:44,248::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 00:04:54,249::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 00:04:54,249::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108871::WARNING::2014-10-04 00:05:54,249::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 00:05:54,249::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 00:05:54,249::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 00:06:04,251::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 00:06:04,251::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108872::WARNING::2014-10-04 00:07:04,251::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 00:07:04,251::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 00:07:04,251::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 00:07:14,253::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 00:07:14,253::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108873::WARNING::2014-10-04 00:08:14,252::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 00:08:14,253::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 00:08:14,253::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 00:08:24,254::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 00:08:24,255::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 00:08:24,255::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108874::WARNING::2014-10-04 00:09:24,255::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 00:09:24,255::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 00:09:24,255::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 00:09:34,256::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 00:09:34,257::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108875::WARNING::2014-10-04 00:10:34,256::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 00:10:34,257::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 00:10:34,257::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 00:10:44,258::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 00:10:44,259::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108876::WARNING::2014-10-04 00:11:44,258::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 00:11:44,259::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 00:11:44,259::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 00:11:54,260::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 00:11:54,260::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108877::WARNING::2014-10-04 00:12:54,260::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 00:12:54,261::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 00:12:54,261::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 00:13:04,262::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 00:13:04,262::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-04 00:13:36,732::__init__::232::IOProcess::(_processLogs) (92105) Queuing response >Thread-18::DEBUG::2014-10-04 00:13:36,733::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 00:13:36,733::__init__::232::IOProcess::(_processLogs) (92165) Got request for method 'glob' >Thread-18::DEBUG::2014-10-04 00:13:36,733::__init__::232::IOProcess::(_processLogs) (92106) Queuing response >Thread-18::DEBUG::2014-10-04 00:13:36,733::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 00:13:36,733::__init__::232::IOProcess::(_processLogs) (92166) Got request for method 'glob' >Thread-108878::WARNING::2014-10-04 00:14:04,262::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 00:14:04,263::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 00:14:04,263::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 00:14:14,263::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 00:14:14,264::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 00:14:14,264::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108879::WARNING::2014-10-04 00:15:14,264::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 00:15:14,265::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 00:15:14,265::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 00:15:24,266::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 00:15:24,266::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108880::WARNING::2014-10-04 00:16:24,266::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 00:16:24,267::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 00:16:24,267::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 00:16:34,268::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 00:16:34,268::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108881::WARNING::2014-10-04 00:17:34,268::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 00:17:34,269::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 00:17:34,269::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 00:17:44,270::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 00:17:44,270::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108882::WARNING::2014-10-04 00:18:44,270::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 00:18:44,271::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 00:18:44,271::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 00:18:54,272::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 00:18:54,272::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108883::WARNING::2014-10-04 00:19:54,272::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 00:19:54,273::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 00:19:54,273::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 00:20:04,273::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 00:20:04,274::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 00:20:04,274::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108884::WARNING::2014-10-04 00:21:04,274::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 00:21:04,275::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 00:21:04,275::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 00:21:14,277::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 00:21:14,277::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108885::WARNING::2014-10-04 00:22:14,276::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 00:22:14,277::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 00:22:14,277::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 00:22:24,278::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 00:22:24,279::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108886::WARNING::2014-10-04 00:23:24,278::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 00:23:24,279::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 00:23:24,279::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 00:23:34,280::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 00:23:34,280::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-04 00:23:40,892::__init__::232::IOProcess::(_processLogs) (92107) Queuing response >Thread-18::DEBUG::2014-10-04 00:23:40,893::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 00:23:40,893::__init__::232::IOProcess::(_processLogs) (92167) Got request for method 'glob' >Thread-108887::WARNING::2014-10-04 00:24:34,280::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 00:24:34,281::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 00:24:34,281::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 00:24:44,282::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 00:24:44,282::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108888::WARNING::2014-10-04 00:25:44,282::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 00:25:44,282::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 00:25:44,283::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 00:25:54,283::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 00:25:54,284::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 00:25:54,284::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108889::WARNING::2014-10-04 00:26:54,284::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 00:26:54,285::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 00:26:54,285::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 00:27:04,286::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 00:27:04,286::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108890::WARNING::2014-10-04 00:28:04,286::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 00:28:04,287::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 00:28:04,287::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 00:28:14,288::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 00:28:14,288::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108891::WARNING::2014-10-04 00:29:14,288::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 00:29:14,288::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 00:29:14,288::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 00:29:24,290::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 00:29:24,290::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108892::WARNING::2014-10-04 00:30:24,290::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 00:30:24,290::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 00:30:24,290::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 00:30:34,291::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 00:30:34,291::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108893::WARNING::2014-10-04 00:31:34,291::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 00:31:34,292::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 00:31:34,292::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 00:31:44,292::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 00:31:44,293::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 00:31:44,293::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108894::WARNING::2014-10-04 00:32:44,293::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 00:32:44,294::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 00:32:44,294::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 00:32:54,295::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 00:32:54,295::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-04 00:33:45,052::__init__::232::IOProcess::(_processLogs) (92108) Queuing response >Thread-18::DEBUG::2014-10-04 00:33:45,053::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 00:33:45,053::__init__::232::IOProcess::(_processLogs) (92168) Got request for method 'glob' >Thread-18::DEBUG::2014-10-04 00:33:45,053::__init__::232::IOProcess::(_processLogs) (92109) Queuing response >Thread-18::DEBUG::2014-10-04 00:33:45,053::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 00:33:45,053::__init__::232::IOProcess::(_processLogs) (92169) Got request for method 'glob' >Thread-108895::WARNING::2014-10-04 00:33:54,295::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 00:33:54,295::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 00:33:54,296::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 00:34:04,297::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 00:34:04,297::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108896::WARNING::2014-10-04 00:35:04,297::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 00:35:04,297::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 00:35:04,297::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 00:35:14,298::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 00:35:14,299::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108897::WARNING::2014-10-04 00:36:14,298::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 00:36:14,299::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 00:36:14,299::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 00:36:24,300::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 00:36:24,300::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108898::WARNING::2014-10-04 00:37:24,300::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 00:37:24,301::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 00:37:24,301::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 00:37:34,301::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 00:37:34,302::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 00:37:34,302::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108899::WARNING::2014-10-04 00:38:34,302::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 00:38:34,303::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 00:38:34,303::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 00:38:44,304::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 00:38:44,304::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108900::WARNING::2014-10-04 00:39:44,304::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 00:39:44,304::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 00:39:44,304::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 00:39:54,306::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 00:39:54,306::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108901::WARNING::2014-10-04 00:40:54,306::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 00:40:54,306::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 00:40:54,306::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 00:41:04,307::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 00:41:04,307::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108902::WARNING::2014-10-04 00:42:04,307::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 00:42:04,308::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 00:42:04,308::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 00:42:14,309::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 00:42:14,309::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108903::WARNING::2014-10-04 00:43:14,309::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 00:43:14,310::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 00:43:14,310::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 00:43:24,310::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 00:43:24,312::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 00:43:24,312::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-04 00:43:49,212::__init__::232::IOProcess::(_processLogs) (92110) Queuing response >Thread-18::DEBUG::2014-10-04 00:43:49,213::__init__::232::IOProcess::(_processLogs) (92111) Queuing response >Thread-18::DEBUG::2014-10-04 00:43:49,213::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 00:43:49,213::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 00:43:49,213::__init__::232::IOProcess::(_processLogs) (92170) Got request for method 'glob' >Thread-18::DEBUG::2014-10-04 00:43:49,213::__init__::232::IOProcess::(_processLogs) (92171) Got request for method 'glob' >Thread-108904::WARNING::2014-10-04 00:44:24,312::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 00:44:24,312::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 00:44:24,312::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 00:44:34,313::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 00:44:34,314::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108905::WARNING::2014-10-04 00:45:34,313::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 00:45:34,314::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 00:45:34,314::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 00:45:44,315::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 00:45:44,315::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108906::WARNING::2014-10-04 00:46:44,315::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 00:46:44,316::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 00:46:44,316::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 00:46:54,317::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 00:46:54,317::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108907::WARNING::2014-10-04 00:47:54,317::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 00:47:54,318::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 00:47:54,318::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 00:48:04,319::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 00:48:04,319::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108908::WARNING::2014-10-04 00:49:04,319::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 00:49:04,319::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 00:49:04,320::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 00:49:14,320::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 00:49:14,321::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 00:49:14,321::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108909::WARNING::2014-10-04 00:50:14,321::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 00:50:14,322::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 00:50:14,322::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 00:50:24,323::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 00:50:24,323::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108910::WARNING::2014-10-04 00:51:24,323::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 00:51:24,324::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 00:51:24,324::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 00:51:34,325::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 00:51:34,326::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108911::WARNING::2014-10-04 00:52:34,325::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 00:52:34,326::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 00:52:34,326::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 00:52:44,327::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 00:52:44,327::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108912::WARNING::2014-10-04 00:53:44,327::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 00:53:44,328::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 00:53:44,328::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 00:53:53,372::__init__::232::IOProcess::(_processLogs) (92112) Queuing response >Thread-18::DEBUG::2014-10-04 00:53:53,373::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 00:53:53,373::__init__::232::IOProcess::(_processLogs) (92172) Got request for method 'glob' >Thread-18::DEBUG::2014-10-04 00:53:53,373::__init__::232::IOProcess::(_processLogs) (92113) Queuing response >Thread-18::DEBUG::2014-10-04 00:53:53,373::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 00:53:53,373::__init__::232::IOProcess::(_processLogs) (92173) Got request for method 'glob' >Thread-18::DEBUG::2014-10-04 00:53:54,329::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 00:53:54,329::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108913::WARNING::2014-10-04 00:54:54,329::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 00:54:54,329::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 00:54:54,330::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 00:55:04,330::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 00:55:04,331::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 00:55:04,331::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108914::WARNING::2014-10-04 00:56:04,331::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 00:56:04,332::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 00:56:04,332::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 00:56:14,333::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 00:56:14,333::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108915::WARNING::2014-10-04 00:57:14,333::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 00:57:14,333::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 00:57:14,334::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 00:57:24,335::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 00:57:24,335::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108916::WARNING::2014-10-04 00:58:24,335::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 00:58:24,335::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 00:58:24,336::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 00:58:34,337::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 00:58:34,337::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108917::WARNING::2014-10-04 00:59:34,337::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 00:59:34,337::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 00:59:34,337::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 00:59:44,339::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 00:59:44,339::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108918::WARNING::2014-10-04 01:00:44,338::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 01:00:44,339::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 01:00:44,339::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 01:00:54,340::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 01:00:54,341::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 01:00:54,341::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108919::WARNING::2014-10-04 01:01:54,341::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 01:01:54,341::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 01:01:54,341::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 01:02:04,342::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 01:02:04,343::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108920::WARNING::2014-10-04 01:03:04,342::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 01:03:04,343::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 01:03:04,343::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 01:03:14,344::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 01:03:14,344::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-04 01:03:57,532::__init__::232::IOProcess::(_processLogs) (92115) Queuing response >Thread-18::DEBUG::2014-10-04 01:03:57,533::__init__::232::IOProcess::(_processLogs) (92114) Queuing response >Thread-18::DEBUG::2014-10-04 01:03:57,533::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 01:03:57,533::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 01:03:57,533::__init__::232::IOProcess::(_processLogs) (92175) Got request for method 'glob' >Thread-18::DEBUG::2014-10-04 01:03:57,533::__init__::232::IOProcess::(_processLogs) (92174) Got request for method 'glob' >Thread-108921::WARNING::2014-10-04 01:04:14,344::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 01:04:14,345::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 01:04:14,345::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 01:04:24,346::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 01:04:24,346::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108922::WARNING::2014-10-04 01:05:24,346::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 01:05:24,347::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 01:05:24,347::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 01:05:34,348::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 01:05:34,348::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108923::WARNING::2014-10-04 01:06:34,348::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 01:06:34,349::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 01:06:34,349::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 01:06:44,349::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 01:06:44,350::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 01:06:44,350::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108924::WARNING::2014-10-04 01:07:44,350::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 01:07:44,351::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 01:07:44,351::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 01:07:54,352::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 01:07:54,352::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108925::WARNING::2014-10-04 01:08:54,352::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 01:08:54,353::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 01:08:54,353::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 01:09:04,354::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 01:09:04,354::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108926::WARNING::2014-10-04 01:10:04,354::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 01:10:04,354::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 01:10:04,355::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 01:10:14,356::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 01:10:14,357::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108927::WARNING::2014-10-04 01:11:14,356::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 01:11:14,357::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 01:11:14,357::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 01:11:24,359::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 01:11:24,359::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108928::WARNING::2014-10-04 01:12:24,358::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 01:12:24,359::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 01:12:24,359::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 01:12:34,360::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 01:12:34,361::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 01:12:34,361::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108929::WARNING::2014-10-04 01:13:34,361::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 01:13:34,361::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 01:13:34,361::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 01:13:44,363::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 01:13:44,363::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-04 01:14:01,693::__init__::232::IOProcess::(_processLogs) (92116) Queuing response >Thread-18::DEBUG::2014-10-04 01:14:01,693::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 01:14:01,693::__init__::232::IOProcess::(_processLogs) (92176) Got request for method 'glob' >Thread-108930::WARNING::2014-10-04 01:14:44,363::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 01:14:44,363::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 01:14:44,363::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 01:14:54,365::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 01:14:54,365::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108931::WARNING::2014-10-04 01:15:54,365::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 01:15:54,365::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 01:15:54,365::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 01:16:04,367::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 01:16:04,367::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108932::WARNING::2014-10-04 01:17:04,367::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 01:17:04,367::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 01:17:04,367::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 01:17:14,369::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 01:17:14,369::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108933::WARNING::2014-10-04 01:18:14,369::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 01:18:14,369::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 01:18:14,369::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 01:18:24,370::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 01:18:24,371::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 01:18:24,371::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108934::WARNING::2014-10-04 01:19:24,371::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 01:19:24,372::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 01:19:24,372::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 01:19:34,373::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 01:19:34,373::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108935::WARNING::2014-10-04 01:20:34,373::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 01:20:34,373::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 01:20:34,374::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 01:20:44,375::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 01:20:44,375::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108936::WARNING::2014-10-04 01:21:44,375::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 01:21:44,375::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 01:21:44,376::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 01:21:54,377::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 01:21:54,377::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108937::WARNING::2014-10-04 01:22:54,377::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 01:22:54,377::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 01:22:54,377::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 01:23:04,379::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 01:23:04,379::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108938::WARNING::2014-10-04 01:24:04,378::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 01:24:04,379::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 01:24:04,379::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 01:24:05,853::__init__::232::IOProcess::(_processLogs) (92117) Queuing response >Thread-18::DEBUG::2014-10-04 01:24:05,853::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 01:24:05,853::__init__::232::IOProcess::(_processLogs) (92177) Got request for method 'glob' >Thread-48::DEBUG::2014-10-04 01:24:14,380::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 01:24:14,380::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 01:24:14,381::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108939::WARNING::2014-10-04 01:25:14,380::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 01:25:14,381::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 01:25:14,381::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 01:25:24,382::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 01:25:24,382::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108940::WARNING::2014-10-04 01:26:24,382::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 01:26:24,383::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 01:26:24,383::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 01:26:34,384::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 01:26:34,384::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108941::WARNING::2014-10-04 01:27:34,384::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 01:27:34,385::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 01:27:34,385::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 01:27:44,386::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 01:27:44,386::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108942::WARNING::2014-10-04 01:28:44,386::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 01:28:44,387::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 01:28:44,387::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 01:28:54,388::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 01:28:54,388::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108943::WARNING::2014-10-04 01:29:54,388::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 01:29:54,388::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 01:29:54,389::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 01:30:04,389::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 01:30:04,390::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 01:30:04,390::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108944::WARNING::2014-10-04 01:31:04,390::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 01:31:04,390::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 01:31:04,391::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 01:31:14,392::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 01:31:14,392::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108945::WARNING::2014-10-04 01:32:14,392::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 01:32:14,392::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 01:32:14,392::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 01:32:24,394::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 01:32:24,394::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108946::WARNING::2014-10-04 01:33:24,394::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 01:33:24,394::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 01:33:24,394::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 01:33:34,395::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 01:33:34,396::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-04 01:34:10,012::__init__::232::IOProcess::(_processLogs) (92118) Queuing response >Thread-18::DEBUG::2014-10-04 01:34:10,013::__init__::232::IOProcess::(_processLogs) (92120) Queuing response >Thread-18::DEBUG::2014-10-04 01:34:10,013::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 01:34:10,013::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 01:34:10,013::__init__::232::IOProcess::(_processLogs) (92178) Got request for method 'glob' >Thread-18::DEBUG::2014-10-04 01:34:10,013::__init__::232::IOProcess::(_processLogs) (92179) Got request for method 'glob' >Thread-108947::WARNING::2014-10-04 01:34:34,395::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 01:34:34,396::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 01:34:34,396::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 01:34:44,397::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 01:34:44,397::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108948::WARNING::2014-10-04 01:35:44,397::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 01:35:44,398::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 01:35:44,398::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 01:35:54,398::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 01:35:54,399::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 01:35:54,399::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108949::WARNING::2014-10-04 01:36:54,399::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 01:36:54,400::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 01:36:54,400::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 01:37:04,401::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 01:37:04,401::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108950::WARNING::2014-10-04 01:38:04,401::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 01:38:04,401::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 01:38:04,401::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 01:38:14,403::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 01:38:14,403::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108951::WARNING::2014-10-04 01:39:14,403::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 01:39:14,403::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 01:39:14,403::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 01:39:24,404::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 01:39:24,405::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108952::WARNING::2014-10-04 01:40:24,404::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 01:40:24,405::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 01:40:24,405::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 01:40:34,406::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 01:40:34,406::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108953::WARNING::2014-10-04 01:41:34,406::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 01:41:34,407::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 01:41:34,407::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 01:41:44,407::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 01:41:44,408::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 01:41:44,408::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108954::WARNING::2014-10-04 01:42:44,408::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 01:42:44,409::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 01:42:44,409::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 01:42:54,410::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 01:42:54,410::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108955::WARNING::2014-10-04 01:43:54,410::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 01:43:54,411::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 01:43:54,411::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 01:44:04,412::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 01:44:04,412::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-04 01:44:16,220::__init__::232::IOProcess::(_processLogs) (92119) Queuing response >Thread-18::DEBUG::2014-10-04 01:44:16,221::__init__::232::IOProcess::(_processLogs) (92122) Queuing response >Thread-18::DEBUG::2014-10-04 01:44:16,221::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 01:44:16,221::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 01:44:16,221::__init__::232::IOProcess::(_processLogs) (92180) Got request for method 'glob' >Thread-18::DEBUG::2014-10-04 01:44:16,221::__init__::232::IOProcess::(_processLogs) (92181) Got request for method 'glob' >Thread-108956::WARNING::2014-10-04 01:45:04,412::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 01:45:04,413::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 01:45:04,413::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 01:45:14,414::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 01:45:14,414::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108957::WARNING::2014-10-04 01:46:14,414::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 01:46:14,415::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 01:46:14,415::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 01:46:24,416::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 01:46:24,416::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108958::WARNING::2014-10-04 01:47:24,416::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 01:47:24,417::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 01:47:24,417::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 01:47:34,417::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 01:47:34,418::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 01:47:34,419::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108959::WARNING::2014-10-04 01:48:34,418::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 01:48:34,419::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 01:48:34,419::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 01:48:44,420::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 01:48:44,421::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108960::WARNING::2014-10-04 01:49:44,420::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 01:49:44,421::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 01:49:44,421::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 01:49:54,422::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 01:49:54,422::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108961::WARNING::2014-10-04 01:50:54,422::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 01:50:54,423::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 01:50:54,423::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 01:51:04,424::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 01:51:04,425::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108962::WARNING::2014-10-04 01:52:04,424::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 01:52:04,425::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 01:52:04,425::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 01:52:14,426::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 01:52:14,427::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108963::WARNING::2014-10-04 01:53:14,426::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 01:53:14,427::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 01:53:14,427::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 01:53:24,427::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 01:53:24,428::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 01:53:24,429::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-04 01:54:22,428::__init__::232::IOProcess::(_processLogs) (92123) Queuing response >Thread-18::DEBUG::2014-10-04 01:54:22,429::__init__::232::IOProcess::(_processLogs) (92121) Queuing response >Thread-18::DEBUG::2014-10-04 01:54:22,429::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 01:54:22,429::__init__::232::IOProcess::(_processLogs) (92182) Got request for method 'glob' >Thread-18::DEBUG::2014-10-04 01:54:22,429::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 01:54:22,429::__init__::232::IOProcess::(_processLogs) (92183) Got request for method 'glob' >Thread-108964::WARNING::2014-10-04 01:54:24,428::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 01:54:24,429::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 01:54:24,429::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 01:54:34,430::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 01:54:34,430::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108965::WARNING::2014-10-04 01:55:34,430::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 01:55:34,431::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 01:55:34,431::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 01:55:44,432::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 01:55:44,432::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108966::WARNING::2014-10-04 01:56:44,432::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 01:56:44,433::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 01:56:44,433::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 01:56:54,434::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 01:56:54,434::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108967::WARNING::2014-10-04 01:57:54,434::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 01:57:54,434::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 01:57:54,435::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 01:58:04,436::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 01:58:04,436::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108968::WARNING::2014-10-04 01:59:04,436::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 01:59:04,436::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 01:59:04,436::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 01:59:14,437::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 01:59:14,438::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 01:59:14,438::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108969::WARNING::2014-10-04 02:00:14,438::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 02:00:14,439::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 02:00:14,439::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 02:00:24,440::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 02:00:24,440::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108970::WARNING::2014-10-04 02:01:24,440::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 02:01:24,441::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 02:01:24,441::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 02:01:34,442::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 02:01:34,442::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108971::WARNING::2014-10-04 02:02:34,442::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 02:02:34,442::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 02:02:34,443::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 02:02:44,444::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 02:02:44,444::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108972::WARNING::2014-10-04 02:03:44,444::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 02:03:44,444::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 02:03:44,445::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 02:03:54,446::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 02:03:54,446::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-04 02:04:26,588::__init__::232::IOProcess::(_processLogs) (92124) Queuing response >Thread-18::DEBUG::2014-10-04 02:04:26,589::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 02:04:26,589::__init__::232::IOProcess::(_processLogs) (92184) Got request for method 'glob' >Thread-108973::WARNING::2014-10-04 02:04:54,446::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 02:04:54,446::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 02:04:54,447::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 02:05:04,447::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 02:05:04,448::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 02:05:04,448::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108974::WARNING::2014-10-04 02:06:04,448::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 02:06:04,448::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 02:06:04,449::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 02:06:14,450::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 02:06:14,450::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108975::WARNING::2014-10-04 02:07:14,450::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 02:07:14,450::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 02:07:14,450::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 02:07:24,452::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 02:07:24,452::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108976::WARNING::2014-10-04 02:08:24,451::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 02:08:24,452::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 02:08:24,452::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 02:08:34,453::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 02:08:34,453::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108977::WARNING::2014-10-04 02:09:34,453::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 02:09:34,454::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 02:09:34,454::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 02:09:44,455::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 02:09:44,455::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108978::WARNING::2014-10-04 02:10:44,455::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 02:10:44,456::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 02:10:44,456::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 02:10:54,456::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 02:10:54,457::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 02:10:54,457::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108979::WARNING::2014-10-04 02:11:54,457::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 02:11:54,458::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 02:11:54,458::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 02:12:04,459::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 02:12:04,459::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108980::WARNING::2014-10-04 02:13:04,459::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 02:13:04,460::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 02:13:04,460::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 02:13:14,461::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 02:13:14,461::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108981::WARNING::2014-10-04 02:14:14,461::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 02:14:14,462::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 02:14:14,462::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 02:14:24,463::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 02:14:24,463::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-04 02:14:30,748::__init__::232::IOProcess::(_processLogs) (92125) Queuing response >Thread-18::DEBUG::2014-10-04 02:14:30,749::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 02:14:30,749::__init__::232::IOProcess::(_processLogs) (92185) Got request for method 'glob' >Thread-18::DEBUG::2014-10-04 02:14:30,749::__init__::232::IOProcess::(_processLogs) (92126) Queuing response >Thread-18::DEBUG::2014-10-04 02:14:30,749::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 02:14:30,749::__init__::232::IOProcess::(_processLogs) (92186) Got request for method 'glob' >Thread-108982::WARNING::2014-10-04 02:15:24,463::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 02:15:24,464::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 02:15:24,464::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 02:15:34,465::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 02:15:34,465::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108983::WARNING::2014-10-04 02:16:34,465::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 02:16:34,466::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 02:16:34,466::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 02:16:44,466::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 02:16:44,467::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 02:16:44,467::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108984::WARNING::2014-10-04 02:17:44,467::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 02:17:44,468::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 02:17:44,468::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 02:17:54,469::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 02:17:54,470::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108985::WARNING::2014-10-04 02:18:54,469::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 02:18:54,470::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 02:18:54,470::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 02:19:04,471::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 02:19:04,472::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108986::WARNING::2014-10-04 02:20:04,471::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 02:20:04,472::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 02:20:04,472::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 02:20:14,473::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 02:20:14,474::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108987::WARNING::2014-10-04 02:21:14,473::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 02:21:14,474::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 02:21:14,474::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 02:21:24,475::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 02:21:24,476::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108988::WARNING::2014-10-04 02:22:24,475::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 02:22:24,476::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 02:22:24,476::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 02:22:34,477::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 02:22:34,478::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 02:22:34,478::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108989::WARNING::2014-10-04 02:23:34,478::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 02:23:34,478::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 02:23:34,478::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 02:23:44,479::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 02:23:44,480::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-04 02:24:34,908::__init__::232::IOProcess::(_processLogs) (92127) Queuing response >Thread-18::DEBUG::2014-10-04 02:24:34,909::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 02:24:34,909::__init__::232::IOProcess::(_processLogs) (92187) Got request for method 'glob' >Thread-108990::WARNING::2014-10-04 02:24:44,479::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 02:24:44,480::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 02:24:44,480::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 02:24:54,481::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 02:24:54,481::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108991::WARNING::2014-10-04 02:25:54,481::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 02:25:54,482::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 02:25:54,482::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 02:26:04,483::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 02:26:04,483::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108992::WARNING::2014-10-04 02:27:04,483::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 02:27:04,483::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 02:27:04,484::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 02:27:14,485::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 02:27:14,485::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108993::WARNING::2014-10-04 02:28:14,485::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 02:28:14,485::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 02:28:14,485::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 02:28:24,486::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 02:28:24,487::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 02:28:24,487::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108994::WARNING::2014-10-04 02:29:24,487::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 02:29:24,487::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 02:29:24,487::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 02:29:34,489::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 02:29:34,489::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108995::WARNING::2014-10-04 02:30:34,489::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 02:30:34,489::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 02:30:34,489::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 02:30:44,490::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 02:30:44,491::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108996::WARNING::2014-10-04 02:31:44,490::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 02:31:44,491::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 02:31:44,491::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 02:31:54,492::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 02:31:54,492::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108997::WARNING::2014-10-04 02:32:54,492::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 02:32:54,493::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 02:32:54,493::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 02:33:04,494::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 02:33:04,494::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-108998::WARNING::2014-10-04 02:34:04,494::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 02:34:04,495::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 02:34:04,495::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 02:34:14,495::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 02:34:14,496::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 02:34:14,496::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-04 02:34:39,068::__init__::232::IOProcess::(_processLogs) (92128) Queuing response >Thread-18::DEBUG::2014-10-04 02:34:39,069::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 02:34:39,069::__init__::232::IOProcess::(_processLogs) (92188) Got request for method 'glob' >Thread-18::DEBUG::2014-10-04 02:34:39,069::__init__::232::IOProcess::(_processLogs) (92129) Queuing response >Thread-18::DEBUG::2014-10-04 02:34:39,069::__init__::232::IOProcess::(_processLogs) (92130) Queuing response >Thread-18::DEBUG::2014-10-04 02:34:39,069::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 02:34:39,069::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 02:34:39,069::__init__::232::IOProcess::(_processLogs) (92189) Got request for method 'glob' >Thread-18::DEBUG::2014-10-04 02:34:39,070::__init__::232::IOProcess::(_processLogs) (92190) Got request for method 'glob' >Thread-108999::WARNING::2014-10-04 02:35:14,496::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 02:35:14,497::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 02:35:14,497::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 02:35:24,498::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 02:35:24,498::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109000::WARNING::2014-10-04 02:36:24,498::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 02:36:24,498::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 02:36:24,499::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 02:36:34,500::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 02:36:34,500::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109001::WARNING::2014-10-04 02:37:34,500::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 02:37:34,500::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 02:37:34,501::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 02:37:44,502::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 02:37:44,502::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109002::WARNING::2014-10-04 02:38:44,502::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 02:38:44,502::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 02:38:44,502::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 02:38:54,504::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 02:38:54,504::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109003::WARNING::2014-10-04 02:39:54,503::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 02:39:54,504::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 02:39:54,504::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 02:40:04,505::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 02:40:04,505::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 02:40:04,506::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109004::WARNING::2014-10-04 02:41:04,505::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 02:41:04,506::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 02:41:04,506::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 02:41:14,507::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 02:41:14,508::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109005::WARNING::2014-10-04 02:42:14,507::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 02:42:14,508::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 02:42:14,508::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 02:42:24,509::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 02:42:24,509::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109006::WARNING::2014-10-04 02:43:24,509::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 02:43:24,510::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 02:43:24,510::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 02:43:34,511::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 02:43:34,511::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109007::WARNING::2014-10-04 02:44:34,511::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 02:44:34,512::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 02:44:34,512::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 02:44:43,228::__init__::232::IOProcess::(_processLogs) (92131) Queuing response >Thread-18::DEBUG::2014-10-04 02:44:43,229::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 02:44:43,229::__init__::232::IOProcess::(_processLogs) (92191) Got request for method 'glob' >Thread-18::DEBUG::2014-10-04 02:44:43,229::__init__::232::IOProcess::(_processLogs) (92132) Queuing response >Thread-18::DEBUG::2014-10-04 02:44:43,229::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 02:44:43,229::__init__::232::IOProcess::(_processLogs) (92192) Got request for method 'glob' >Thread-18::DEBUG::2014-10-04 02:44:44,513::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 02:44:44,513::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109008::WARNING::2014-10-04 02:45:44,513::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 02:45:44,514::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 02:45:44,514::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 02:45:54,514::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 02:45:54,515::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 02:45:54,516::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109009::WARNING::2014-10-04 02:46:54,515::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 02:46:54,516::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 02:46:54,516::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 02:47:04,517::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 02:47:04,518::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109010::WARNING::2014-10-04 02:48:04,517::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 02:48:04,518::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 02:48:04,518::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 02:48:14,519::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 02:48:14,520::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109011::WARNING::2014-10-04 02:49:14,519::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 02:49:14,520::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 02:49:14,520::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 02:49:24,521::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 02:49:24,522::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109012::WARNING::2014-10-04 02:50:24,521::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 02:50:24,522::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 02:50:24,522::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 02:50:34,523::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 02:50:34,523::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109013::WARNING::2014-10-04 02:51:34,523::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 02:51:34,524::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 02:51:34,524::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 02:51:44,524::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 02:51:44,525::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 02:51:44,526::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109014::WARNING::2014-10-04 02:52:44,525::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 02:52:44,526::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 02:52:44,526::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 02:52:54,527::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 02:52:54,527::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109015::WARNING::2014-10-04 02:53:54,527::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 02:53:54,528::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 02:53:54,528::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 02:54:04,529::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 02:54:04,529::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-04 02:54:47,388::__init__::232::IOProcess::(_processLogs) (92133) Queuing response >Thread-18::DEBUG::2014-10-04 02:54:47,389::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 02:54:47,389::__init__::232::IOProcess::(_processLogs) (92193) Got request for method 'glob' >Thread-18::DEBUG::2014-10-04 02:54:47,389::__init__::232::IOProcess::(_processLogs) (92134) Queuing response >Thread-18::DEBUG::2014-10-04 02:54:47,389::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 02:54:47,389::__init__::232::IOProcess::(_processLogs) (92194) Got request for method 'glob' >Thread-109016::WARNING::2014-10-04 02:55:04,529::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 02:55:04,530::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 02:55:04,530::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 02:55:14,531::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 02:55:14,531::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109017::WARNING::2014-10-04 02:56:14,531::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 02:56:14,532::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 02:56:14,532::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 02:56:24,533::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 02:56:24,533::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109018::WARNING::2014-10-04 02:57:24,533::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 02:57:24,533::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 02:57:24,534::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 02:57:34,534::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 02:57:34,535::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 02:57:34,535::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109019::WARNING::2014-10-04 02:58:34,535::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 02:58:34,535::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 02:58:34,536::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 02:58:44,537::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 02:58:44,537::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109020::WARNING::2014-10-04 02:59:44,537::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 02:59:44,537::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 02:59:44,537::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 02:59:54,539::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 02:59:54,539::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109021::WARNING::2014-10-04 03:00:54,539::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 03:00:54,539::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 03:00:54,539::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 03:01:04,540::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 03:01:04,541::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109022::WARNING::2014-10-04 03:02:04,540::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 03:02:04,541::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 03:02:04,541::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 03:02:14,543::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 03:02:14,543::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109023::WARNING::2014-10-04 03:03:14,542::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 03:03:14,543::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 03:03:14,543::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 03:03:24,544::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 03:03:24,545::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 03:03:24,545::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109024::WARNING::2014-10-04 03:04:24,545::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 03:04:24,545::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 03:04:24,545::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 03:04:34,547::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 03:04:34,547::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-04 03:04:51,548::__init__::232::IOProcess::(_processLogs) (92135) Queuing response >Thread-18::DEBUG::2014-10-04 03:04:51,549::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 03:04:51,549::__init__::232::IOProcess::(_processLogs) (92195) Got request for method 'glob' >Thread-18::DEBUG::2014-10-04 03:04:51,549::__init__::232::IOProcess::(_processLogs) (92136) Queuing response >Thread-18::DEBUG::2014-10-04 03:04:51,549::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 03:04:51,549::__init__::232::IOProcess::(_processLogs) (92196) Got request for method 'glob' >Thread-109025::WARNING::2014-10-04 03:05:34,546::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 03:05:34,547::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 03:05:34,547::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 03:05:44,548::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 03:05:44,549::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109026::WARNING::2014-10-04 03:06:44,548::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 03:06:44,549::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 03:06:44,549::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 03:06:54,550::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 03:06:54,550::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109027::WARNING::2014-10-04 03:07:54,550::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 03:07:54,551::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 03:07:54,551::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 03:08:04,552::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 03:08:04,552::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109028::WARNING::2014-10-04 03:09:04,552::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 03:09:04,553::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 03:09:04,553::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 03:09:14,553::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 03:09:14,554::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 03:09:14,555::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109029::WARNING::2014-10-04 03:10:14,554::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 03:10:14,555::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 03:10:14,555::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 03:10:24,556::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 03:10:24,556::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109030::WARNING::2014-10-04 03:11:24,556::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 03:11:24,557::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 03:11:24,557::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 03:11:34,558::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 03:11:34,558::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109031::WARNING::2014-10-04 03:12:34,558::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 03:12:34,559::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 03:12:34,559::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 03:12:44,560::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 03:12:44,560::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109032::WARNING::2014-10-04 03:13:44,560::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 03:13:44,561::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 03:13:44,561::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 03:13:54,562::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 03:13:54,562::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109033::WARNING::2014-10-04 03:14:54,562::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 03:14:54,563::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 03:14:54,563::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 03:14:55,708::__init__::232::IOProcess::(_processLogs) (92137) Queuing response >Thread-18::DEBUG::2014-10-04 03:14:55,709::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 03:14:55,709::__init__::232::IOProcess::(_processLogs) (92197) Got request for method 'glob' >Thread-18::DEBUG::2014-10-04 03:14:55,709::__init__::232::IOProcess::(_processLogs) (92138) Queuing response >Thread-18::DEBUG::2014-10-04 03:14:55,709::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 03:14:55,709::__init__::232::IOProcess::(_processLogs) (92139) Queuing response >Thread-18::DEBUG::2014-10-04 03:14:55,709::__init__::232::IOProcess::(_processLogs) (92198) Got request for method 'glob' >Thread-18::DEBUG::2014-10-04 03:14:55,709::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 03:14:55,709::__init__::232::IOProcess::(_processLogs) (92199) Got request for method 'glob' >Thread-48::DEBUG::2014-10-04 03:15:04,563::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 03:15:04,564::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 03:15:04,564::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109034::WARNING::2014-10-04 03:16:04,564::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 03:16:04,565::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 03:16:04,565::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 03:16:14,566::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 03:16:14,566::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109035::WARNING::2014-10-04 03:17:14,566::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 03:17:14,567::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 03:17:14,567::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 03:17:24,568::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 03:17:24,569::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109036::WARNING::2014-10-04 03:18:24,568::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 03:18:24,569::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 03:18:24,569::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 03:18:34,570::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 03:18:34,571::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109037::WARNING::2014-10-04 03:19:34,570::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 03:19:34,571::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 03:19:34,571::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 03:19:44,572::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 03:19:44,573::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109038::WARNING::2014-10-04 03:20:44,572::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 03:20:44,573::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 03:20:44,573::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 03:20:54,574::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 03:20:54,575::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 03:20:54,575::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109039::WARNING::2014-10-04 03:21:54,575::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 03:21:54,575::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 03:21:54,575::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 03:22:04,577::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 03:22:04,577::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109040::WARNING::2014-10-04 03:23:04,576::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 03:23:04,577::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 03:23:04,577::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 03:23:14,578::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 03:23:14,579::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109041::WARNING::2014-10-04 03:24:14,578::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 03:24:14,579::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 03:24:14,579::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 03:24:24,580::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 03:24:24,580::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-04 03:24:59,868::__init__::232::IOProcess::(_processLogs) (92140) Queuing response >Thread-18::DEBUG::2014-10-04 03:24:59,869::__init__::232::IOProcess::(_processLogs) (92141) Queuing response >Thread-18::DEBUG::2014-10-04 03:24:59,869::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 03:24:59,869::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 03:24:59,869::__init__::232::IOProcess::(_processLogs) (92200) Got request for method 'glob' >Thread-18::DEBUG::2014-10-04 03:24:59,869::__init__::232::IOProcess::(_processLogs) (92201) Got request for method 'glob' >Thread-109042::WARNING::2014-10-04 03:25:24,580::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 03:25:24,581::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 03:25:24,581::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 03:25:34,582::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 03:25:34,582::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109043::WARNING::2014-10-04 03:26:34,582::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 03:26:34,582::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 03:26:34,582::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 03:26:44,583::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 03:26:44,584::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 03:26:44,584::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109044::WARNING::2014-10-04 03:27:44,584::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 03:27:44,584::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 03:27:44,584::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 03:27:54,585::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 03:27:54,586::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109045::WARNING::2014-10-04 03:28:54,585::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 03:28:54,586::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 03:28:54,586::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 03:29:04,587::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 03:29:04,587::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109046::WARNING::2014-10-04 03:30:04,587::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 03:30:04,588::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 03:30:04,588::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 03:30:14,589::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 03:30:14,589::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109047::WARNING::2014-10-04 03:31:14,589::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 03:31:14,590::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 03:31:14,590::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 03:31:24,591::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 03:31:24,591::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109048::WARNING::2014-10-04 03:32:24,591::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 03:32:24,591::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 03:32:24,592::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 03:32:34,592::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 03:32:34,593::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 03:32:34,593::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109049::WARNING::2014-10-04 03:33:34,593::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 03:33:34,593::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 03:33:34,593::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 03:33:44,595::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 03:33:44,595::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109050::WARNING::2014-10-04 03:34:44,595::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 03:34:44,595::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 03:34:44,595::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 03:34:54,596::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 03:34:54,597::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-04 03:35:06,076::__init__::232::IOProcess::(_processLogs) (92142) Queuing response >Thread-18::DEBUG::2014-10-04 03:35:06,077::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 03:35:06,077::__init__::232::IOProcess::(_processLogs) (92202) Got request for method 'glob' >Thread-109051::WARNING::2014-10-04 03:35:54,596::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 03:35:54,597::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 03:35:54,597::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 03:36:04,598::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 03:36:04,598::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109052::WARNING::2014-10-04 03:37:04,598::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 03:37:04,599::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 03:37:04,599::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 03:37:14,600::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 03:37:14,600::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109053::WARNING::2014-10-04 03:38:14,600::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 03:38:14,601::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 03:38:14,601::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 03:38:24,601::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 03:38:24,602::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 03:38:24,603::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109054::WARNING::2014-10-04 03:39:24,602::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 03:39:24,603::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 03:39:24,603::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 03:39:34,604::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 03:39:34,604::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109055::WARNING::2014-10-04 03:40:34,604::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 03:40:34,605::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 03:40:34,605::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 03:40:44,606::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 03:40:44,606::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109056::WARNING::2014-10-04 03:41:44,606::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 03:41:44,606::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 03:41:44,607::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 03:41:54,608::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 03:41:54,608::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109057::WARNING::2014-10-04 03:42:54,608::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 03:42:54,608::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 03:42:54,609::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 03:43:04,610::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 03:43:04,610::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109058::WARNING::2014-10-04 03:44:04,610::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 03:44:04,610::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 03:44:04,611::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 03:44:14,611::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 03:44:14,612::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 03:44:14,612::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-04 03:45:10,236::__init__::232::IOProcess::(_processLogs) (92143) Queuing response >Thread-18::DEBUG::2014-10-04 03:45:10,237::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 03:45:10,237::__init__::232::IOProcess::(_processLogs) (92203) Got request for method 'glob' >Thread-109059::WARNING::2014-10-04 03:45:14,612::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 03:45:14,612::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 03:45:14,613::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 03:45:24,614::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 03:45:24,614::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109060::WARNING::2014-10-04 03:46:24,614::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 03:46:24,614::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 03:46:24,614::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 03:46:34,616::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 03:46:34,616::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109061::WARNING::2014-10-04 03:47:34,616::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 03:47:34,616::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 03:47:34,616::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 03:47:44,618::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 03:47:44,618::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109062::WARNING::2014-10-04 03:48:44,618::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 03:48:44,618::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 03:48:44,618::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 03:48:54,620::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 03:48:54,620::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109063::WARNING::2014-10-04 03:49:54,620::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 03:49:54,620::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 03:49:54,620::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 03:50:04,621::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 03:50:04,622::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 03:50:04,622::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109064::WARNING::2014-10-04 03:51:04,622::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 03:51:04,622::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 03:51:04,623::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 03:51:14,624::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 03:51:14,624::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109065::WARNING::2014-10-04 03:52:14,624::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 03:52:14,625::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 03:52:14,626::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 03:52:24,627::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 03:52:24,627::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109066::WARNING::2014-10-04 03:53:24,627::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 03:53:24,627::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 03:53:24,628::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 03:53:34,629::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 03:53:34,629::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109067::WARNING::2014-10-04 03:54:34,629::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 03:54:34,629::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 03:54:34,630::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 03:54:44,631::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 03:54:44,631::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109068::WARNING::2014-10-04 03:55:44,631::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 03:55:44,631::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 03:55:44,632::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 03:55:54,632::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 03:55:54,633::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 03:55:54,633::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109069::WARNING::2014-10-04 03:56:54,633::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 03:56:54,633::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 03:56:54,634::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 03:57:04,635::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 03:57:04,635::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109070::WARNING::2014-10-04 03:58:04,635::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 03:58:04,635::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 03:58:04,635::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 03:58:14,637::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 03:58:14,637::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109071::WARNING::2014-10-04 03:59:14,636::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 03:59:14,637::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 03:59:14,637::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 03:59:24,638::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 03:59:24,639::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109072::WARNING::2014-10-04 04:00:24,638::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 04:00:24,639::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 04:00:24,639::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 04:00:34,640::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 04:00:34,640::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109073::WARNING::2014-10-04 04:01:34,640::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 04:01:34,641::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 04:01:34,641::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 04:01:44,641::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 04:01:44,642::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 04:01:44,642::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109074::WARNING::2014-10-04 04:02:44,642::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 04:02:44,643::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 04:02:44,643::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 04:02:54,644::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 04:02:54,644::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109075::WARNING::2014-10-04 04:03:54,644::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 04:03:54,644::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 04:03:54,645::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 04:04:04,646::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 04:04:04,647::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109076::WARNING::2014-10-04 04:05:04,646::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 04:05:04,646::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 04:05:04,646::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 04:05:14,648::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 04:05:14,648::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109077::WARNING::2014-10-04 04:06:14,647::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 04:06:14,648::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 04:06:14,648::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 04:06:24,650::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 04:06:24,650::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109078::WARNING::2014-10-04 04:07:24,649::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 04:07:24,650::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 04:07:24,650::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 04:07:34,650::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 04:07:34,651::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 04:07:34,652::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109079::WARNING::2014-10-04 04:08:34,651::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 04:08:34,652::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 04:08:34,652::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 04:08:44,653::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 04:08:44,654::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109080::WARNING::2014-10-04 04:09:44,653::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 04:09:44,654::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 04:09:44,654::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 04:09:54,655::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 04:09:54,655::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109081::WARNING::2014-10-04 04:10:54,655::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 04:10:54,656::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 04:10:54,656::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 04:11:04,657::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 04:11:04,657::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109082::WARNING::2014-10-04 04:12:04,657::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 04:12:04,657::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 04:12:04,658::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 04:12:14,659::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 04:12:14,659::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109083::WARNING::2014-10-04 04:13:14,659::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 04:13:14,659::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 04:13:14,659::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 04:13:24,660::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 04:13:24,661::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 04:13:24,661::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109084::WARNING::2014-10-04 04:14:24,661::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 04:14:24,661::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 04:14:24,662::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 04:14:34,663::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 04:14:34,663::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109085::WARNING::2014-10-04 04:15:34,663::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 04:15:34,663::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 04:15:34,663::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 04:15:44,665::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 04:15:44,665::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109086::WARNING::2014-10-04 04:16:44,664::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 04:16:44,665::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 04:16:44,665::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 04:16:54,666::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 04:16:54,667::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109087::WARNING::2014-10-04 04:17:54,666::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 04:17:54,667::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 04:17:54,667::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 04:18:04,668::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 04:18:04,669::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109088::WARNING::2014-10-04 04:19:04,668::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 04:19:04,669::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 04:19:04,669::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 04:19:14,670::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 04:19:14,671::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 04:19:14,671::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109089::WARNING::2014-10-04 04:20:14,671::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 04:20:14,672::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 04:20:14,672::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 04:20:24,673::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 04:20:24,673::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109090::WARNING::2014-10-04 04:21:24,673::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 04:21:24,674::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 04:21:24,674::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 04:21:34,675::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 04:21:34,675::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109091::WARNING::2014-10-04 04:22:34,675::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 04:22:34,676::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 04:22:34,676::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 04:22:44,677::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 04:22:44,677::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109092::WARNING::2014-10-04 04:23:44,677::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 04:23:44,677::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 04:23:44,678::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 04:23:54,679::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 04:23:54,679::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109093::WARNING::2014-10-04 04:24:54,679::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 04:24:54,679::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 04:24:54,679::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 04:25:04,680::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 04:25:04,681::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 04:25:04,681::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109094::WARNING::2014-10-04 04:26:04,681::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 04:26:04,681::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 04:26:04,681::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 04:26:14,682::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 04:26:14,683::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109095::WARNING::2014-10-04 04:27:14,682::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 04:27:14,683::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 04:27:14,683::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 04:27:24,684::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 04:27:24,684::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109096::WARNING::2014-10-04 04:28:24,684::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 04:28:24,685::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 04:28:24,685::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 04:28:34,686::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 04:28:34,686::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109097::WARNING::2014-10-04 04:29:34,686::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 04:29:34,686::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 04:29:34,687::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 04:29:44,688::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 04:29:44,688::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109098::WARNING::2014-10-04 04:30:44,688::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 04:30:44,688::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 04:30:44,689::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 04:30:54,689::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 04:30:54,690::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 04:30:54,690::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109099::WARNING::2014-10-04 04:31:54,690::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 04:31:54,690::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 04:31:54,691::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 04:32:04,692::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 04:32:04,692::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109100::WARNING::2014-10-04 04:33:04,692::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 04:33:04,692::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 04:33:04,692::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 04:33:14,694::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 04:33:14,694::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109101::WARNING::2014-10-04 04:34:14,694::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 04:34:14,694::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 04:34:14,694::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 04:34:24,696::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 04:34:24,696::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109102::WARNING::2014-10-04 04:35:24,695::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 04:35:24,696::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 04:35:24,696::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 04:35:34,697::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 04:35:34,698::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109103::WARNING::2014-10-04 04:36:34,697::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 04:36:34,698::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 04:36:34,698::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 04:36:44,699::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 04:36:44,700::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 04:36:44,700::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109104::WARNING::2014-10-04 04:37:44,699::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 04:37:44,700::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 04:37:44,700::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 04:37:54,701::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 04:37:54,702::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109105::WARNING::2014-10-04 04:38:54,701::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 04:38:54,702::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 04:38:54,702::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 04:39:04,703::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 04:39:04,703::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109106::WARNING::2014-10-04 04:40:04,703::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 04:40:04,704::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 04:40:04,704::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 04:40:14,705::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 04:40:14,705::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109107::WARNING::2014-10-04 04:41:14,705::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 04:41:14,705::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 04:41:14,706::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 04:41:24,707::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 04:41:24,707::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109108::WARNING::2014-10-04 04:42:24,707::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 04:42:24,707::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 04:42:24,708::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 04:42:34,708::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 04:42:34,709::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 04:42:34,709::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109109::WARNING::2014-10-04 04:43:34,709::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 04:43:34,709::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 04:43:34,710::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 04:43:44,711::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 04:43:44,711::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109110::WARNING::2014-10-04 04:44:44,711::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 04:44:44,711::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 04:44:44,712::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 04:44:54,713::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 04:44:54,713::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109111::WARNING::2014-10-04 04:45:54,713::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 04:45:54,713::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 04:45:54,713::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 04:46:04,715::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 04:46:04,715::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109112::WARNING::2014-10-04 04:47:04,715::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 04:47:04,715::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 04:47:04,715::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 04:47:14,717::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 04:47:14,717::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109113::WARNING::2014-10-04 04:48:14,716::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 04:48:14,717::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 04:48:14,717::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 04:48:24,718::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 04:48:24,719::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 04:48:24,719::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109114::WARNING::2014-10-04 04:49:24,719::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 04:49:24,719::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 04:49:24,719::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 04:49:34,721::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 04:49:34,721::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109115::WARNING::2014-10-04 04:50:34,720::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 04:50:34,721::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 04:50:34,721::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 04:50:44,723::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 04:50:44,723::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109116::WARNING::2014-10-04 04:51:44,723::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 04:51:44,723::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 04:51:44,723::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 04:51:54,725::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 04:51:54,725::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109117::WARNING::2014-10-04 04:52:54,724::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 04:52:54,725::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 04:52:54,725::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 04:53:04,726::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 04:53:04,726::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109118::WARNING::2014-10-04 04:54:04,726::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 04:54:04,727::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 04:54:04,727::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 04:54:14,727::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 04:54:14,728::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 04:54:14,728::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109119::WARNING::2014-10-04 04:55:14,728::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 04:55:14,729::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 04:55:14,729::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 04:55:24,730::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 04:55:24,730::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109120::WARNING::2014-10-04 04:56:24,730::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 04:56:24,731::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 04:56:24,731::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 04:56:34,732::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 04:56:34,732::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109121::WARNING::2014-10-04 04:57:34,732::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 04:57:34,732::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 04:57:34,733::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 04:57:44,734::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 04:57:44,734::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109122::WARNING::2014-10-04 04:58:44,734::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 04:58:44,734::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 04:58:44,735::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 04:58:54,736::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 04:58:54,736::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109123::WARNING::2014-10-04 04:59:54,736::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 04:59:54,736::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 04:59:54,736::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 05:00:04,737::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 05:00:04,738::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 05:00:04,738::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109124::WARNING::2014-10-04 05:01:04,738::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 05:01:04,738::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 05:01:04,738::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 05:01:14,740::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 05:01:14,740::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109125::WARNING::2014-10-04 05:02:14,740::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 05:02:14,740::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 05:02:14,740::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 05:02:24,741::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 05:02:24,742::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109126::WARNING::2014-10-04 05:03:24,741::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 05:03:24,742::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 05:03:24,742::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 05:03:34,743::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 05:03:34,743::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109127::WARNING::2014-10-04 05:04:34,743::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 05:04:34,744::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 05:04:34,744::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 05:04:44,745::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 05:04:44,745::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109128::WARNING::2014-10-04 05:05:44,745::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 05:05:44,746::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 05:05:44,746::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 05:05:54,746::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 05:05:54,747::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 05:05:54,748::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109129::WARNING::2014-10-04 05:06:54,747::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 05:06:54,748::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 05:06:54,748::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 05:07:04,749::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 05:07:04,749::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109130::WARNING::2014-10-04 05:08:04,749::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 05:08:04,750::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 05:08:04,750::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 05:08:14,751::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 05:08:14,751::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109131::WARNING::2014-10-04 05:09:14,751::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 05:09:14,751::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 05:09:14,752::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 05:09:24,753::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 05:09:24,753::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109132::WARNING::2014-10-04 05:10:24,753::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 05:10:24,753::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 05:10:24,753::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 05:10:34,755::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 05:10:34,755::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109133::WARNING::2014-10-04 05:11:34,755::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 05:11:34,755::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 05:11:34,755::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 05:11:44,756::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 05:11:44,757::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 05:11:44,757::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109134::WARNING::2014-10-04 05:12:44,757::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 05:12:44,757::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 05:12:44,757::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 05:12:54,759::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 05:12:54,759::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109135::WARNING::2014-10-04 05:13:54,758::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 05:13:54,759::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 05:13:54,759::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 05:14:04,760::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 05:14:04,761::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109136::WARNING::2014-10-04 05:15:04,760::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 05:15:04,761::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 05:15:04,761::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 05:15:14,762::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 05:15:14,763::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-04 05:15:51,772::__init__::232::IOProcess::(_processLogs) (92144) Queuing response >Thread-18::DEBUG::2014-10-04 05:15:51,773::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 05:15:51,773::__init__::232::IOProcess::(_processLogs) (92204) Got request for method 'glob' >Thread-109137::WARNING::2014-10-04 05:16:14,762::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 05:16:14,763::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 05:16:14,763::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 05:16:24,764::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 05:16:24,764::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109138::WARNING::2014-10-04 05:17:24,764::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 05:17:24,765::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 05:17:24,765::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 05:17:34,765::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 05:17:34,766::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 05:17:34,766::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109139::WARNING::2014-10-04 05:18:34,766::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 05:18:34,767::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 05:18:34,767::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 05:18:44,768::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 05:18:44,768::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109140::WARNING::2014-10-04 05:19:44,768::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 05:19:44,769::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 05:19:44,769::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 05:19:54,770::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 05:19:54,770::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109141::WARNING::2014-10-04 05:20:54,770::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 05:20:54,771::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 05:20:54,771::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 05:21:04,772::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 05:21:04,772::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109142::WARNING::2014-10-04 05:22:04,772::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 05:22:04,773::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 05:22:04,773::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 05:22:14,774::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 05:22:14,774::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109143::WARNING::2014-10-04 05:23:14,774::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 05:23:14,775::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 05:23:14,775::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 05:23:24,775::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 05:23:24,776::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 05:23:24,776::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109144::WARNING::2014-10-04 05:24:24,776::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 05:24:24,777::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 05:24:24,777::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 05:24:34,778::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 05:24:34,778::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109145::WARNING::2014-10-04 05:25:34,778::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 05:25:34,778::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 05:25:34,779::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 05:25:44,780::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 05:25:44,780::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109146::WARNING::2014-10-04 05:26:44,780::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 05:26:44,780::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 05:26:44,780::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 05:26:54,782::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 05:26:54,782::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109147::WARNING::2014-10-04 05:27:54,781::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 05:27:54,782::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 05:27:54,782::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 05:28:04,783::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 05:28:04,783::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109148::WARNING::2014-10-04 05:29:04,783::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 05:29:04,784::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 05:29:04,784::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 05:29:14,784::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 05:29:14,785::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 05:29:14,785::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109149::WARNING::2014-10-04 05:30:14,785::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 05:30:14,786::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 05:30:14,786::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 05:30:24,787::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 05:30:24,787::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109150::WARNING::2014-10-04 05:31:24,787::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 05:31:24,788::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 05:31:24,788::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 05:31:34,789::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 05:31:34,790::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109151::WARNING::2014-10-04 05:32:34,789::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 05:32:34,790::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 05:32:34,791::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 05:32:44,792::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 05:32:44,792::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109152::WARNING::2014-10-04 05:33:44,792::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 05:33:44,793::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 05:33:44,793::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 05:33:54,794::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 05:33:54,794::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109153::WARNING::2014-10-04 05:34:54,794::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 05:34:54,794::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 05:34:54,795::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 05:35:04,795::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 05:35:04,796::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 05:35:04,796::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109154::WARNING::2014-10-04 05:36:04,796::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 05:36:04,796::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 05:36:04,797::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 05:36:14,798::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 05:36:14,798::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109155::WARNING::2014-10-04 05:37:14,798::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 05:37:14,798::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 05:37:14,799::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 05:37:24,800::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 05:37:24,800::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109156::WARNING::2014-10-04 05:38:24,800::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 05:38:24,800::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 05:38:24,801::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 05:38:34,802::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 05:38:34,802::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109157::WARNING::2014-10-04 05:39:34,802::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 05:39:34,802::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 05:39:34,802::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 05:39:44,803::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 05:39:44,804::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109158::WARNING::2014-10-04 05:40:44,803::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 05:40:44,804::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 05:40:44,804::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 05:40:54,805::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 05:40:54,806::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 05:40:54,806::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109159::WARNING::2014-10-04 05:41:54,805::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 05:41:54,806::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 05:41:54,806::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 05:42:04,808::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 05:42:04,808::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109160::WARNING::2014-10-04 05:43:04,807::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 05:43:04,808::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 05:43:04,808::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 05:43:14,809::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 05:43:14,810::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109161::WARNING::2014-10-04 05:44:14,809::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 05:44:14,810::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 05:44:14,810::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 05:44:24,811::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 05:44:24,811::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109162::WARNING::2014-10-04 05:45:24,811::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 05:45:24,812::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 05:45:24,812::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 05:45:34,813::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 05:45:34,813::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109163::WARNING::2014-10-04 05:46:34,813::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 05:46:34,814::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 05:46:34,814::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 05:46:44,815::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 05:46:44,815::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 05:46:44,816::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109164::WARNING::2014-10-04 05:47:44,815::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 05:47:44,816::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 05:47:44,816::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 05:47:54,817::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 05:47:54,818::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109165::WARNING::2014-10-04 05:48:54,817::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 05:48:54,818::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 05:48:54,818::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 05:49:04,819::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 05:49:04,819::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109166::WARNING::2014-10-04 05:50:04,819::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 05:50:04,820::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 05:50:04,820::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 05:50:14,821::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 05:50:14,821::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109167::WARNING::2014-10-04 05:51:14,821::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 05:51:14,822::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 05:51:14,822::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 05:51:24,823::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 05:51:24,823::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109168::WARNING::2014-10-04 05:52:24,823::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 05:52:24,823::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 05:52:24,824::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 05:52:34,824::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 05:52:34,825::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 05:52:34,825::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109169::WARNING::2014-10-04 05:53:34,825::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 05:53:34,826::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 05:53:34,826::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 05:53:44,827::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 05:53:44,827::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109170::WARNING::2014-10-04 05:54:44,827::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 05:54:44,827::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 05:54:44,828::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 05:54:54,829::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 05:54:54,829::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109171::WARNING::2014-10-04 05:55:54,829::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 05:55:54,829::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 05:55:54,829::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 05:56:04,831::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 05:56:04,831::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-04 05:56:10,460::__init__::232::IOProcess::(_processLogs) (92145) Queuing response >Thread-18::DEBUG::2014-10-04 05:56:10,461::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 05:56:10,461::__init__::232::IOProcess::(_processLogs) (92205) Got request for method 'glob' >Thread-109172::WARNING::2014-10-04 05:57:04,830::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 05:57:04,831::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 05:57:04,831::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 05:57:14,832::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 05:57:14,833::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109173::WARNING::2014-10-04 05:58:14,832::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 05:58:14,833::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 05:58:14,833::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 05:58:24,833::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 05:58:24,834::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 05:58:24,834::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109174::WARNING::2014-10-04 05:59:24,834::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 05:59:24,835::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 05:59:24,835::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 05:59:34,836::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 05:59:34,836::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109175::WARNING::2014-10-04 06:00:34,836::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 06:00:34,837::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 06:00:34,837::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 06:00:44,838::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 06:00:44,838::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109176::WARNING::2014-10-04 06:01:44,838::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 06:01:44,838::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 06:01:44,839::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 06:01:54,840::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 06:01:54,840::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109177::WARNING::2014-10-04 06:02:54,840::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 06:02:54,840::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 06:02:54,840::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 06:03:04,842::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 06:03:04,842::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109178::WARNING::2014-10-04 06:04:04,841::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 06:04:04,842::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 06:04:04,842::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 06:04:14,843::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 06:04:14,844::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 06:04:14,844::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109179::WARNING::2014-10-04 06:05:14,843::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 06:05:14,844::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 06:05:14,844::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 06:05:24,845::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 06:05:24,845::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-04 06:06:14,620::__init__::232::IOProcess::(_processLogs) (92146) Queuing response >Thread-18::DEBUG::2014-10-04 06:06:14,621::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 06:06:14,622::__init__::232::IOProcess::(_processLogs) (92206) Got request for method 'glob' >Thread-109180::WARNING::2014-10-04 06:06:24,845::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 06:06:24,846::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 06:06:24,846::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 06:06:34,847::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 06:06:34,847::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109181::WARNING::2014-10-04 06:07:34,847::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 06:07:34,847::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 06:07:34,848::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 06:07:44,849::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 06:07:44,849::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109182::WARNING::2014-10-04 06:08:44,849::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 06:08:44,849::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 06:08:44,849::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 06:08:54,851::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 06:08:54,851::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109183::WARNING::2014-10-04 06:09:54,850::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 06:09:54,851::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 06:09:54,851::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 06:10:04,852::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 06:10:04,852::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 06:10:04,853::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109184::WARNING::2014-10-04 06:11:04,852::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 06:11:04,853::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 06:11:04,853::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 06:11:14,854::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 06:11:14,854::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109185::WARNING::2014-10-04 06:12:14,854::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 06:12:14,855::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 06:12:14,855::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 06:12:24,856::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 06:12:24,856::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109186::WARNING::2014-10-04 06:13:24,856::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 06:13:24,856::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 06:13:24,857::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 06:13:34,858::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 06:13:34,858::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109187::WARNING::2014-10-04 06:14:34,858::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 06:14:34,858::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 06:14:34,858::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 06:14:44,860::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 06:14:44,860::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109188::WARNING::2014-10-04 06:15:44,859::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 06:15:44,860::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 06:15:44,860::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 06:15:54,860::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 06:15:54,861::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 06:15:54,862::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-04 06:16:18,780::__init__::232::IOProcess::(_processLogs) (92147) Queuing response >Thread-18::DEBUG::2014-10-04 06:16:18,781::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 06:16:18,781::__init__::232::IOProcess::(_processLogs) (92207) Got request for method 'glob' >Thread-109189::WARNING::2014-10-04 06:16:54,861::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 06:16:54,862::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 06:16:54,862::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 06:17:04,863::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 06:17:04,863::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109190::WARNING::2014-10-04 06:18:04,863::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 06:18:04,863::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 06:18:04,864::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 06:18:14,865::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 06:18:14,865::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109191::WARNING::2014-10-04 06:19:14,865::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 06:19:14,865::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 06:19:14,865::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 06:19:24,866::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 06:19:24,867::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109192::WARNING::2014-10-04 06:20:24,866::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 06:20:24,867::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 06:20:24,867::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 06:20:34,869::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 06:20:34,869::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109193::WARNING::2014-10-04 06:21:34,868::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 06:21:34,869::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 06:21:34,869::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 06:21:44,870::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 06:21:44,871::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 06:21:44,871::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109194::WARNING::2014-10-04 06:22:44,871::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 06:22:44,871::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 06:22:44,871::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 06:22:54,873::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 06:22:54,873::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109195::WARNING::2014-10-04 06:23:54,872::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 06:23:54,873::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 06:23:54,873::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 06:24:04,874::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 06:24:04,874::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109196::WARNING::2014-10-04 06:25:04,874::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 06:25:04,875::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 06:25:04,875::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 06:25:14,876::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 06:25:14,876::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109197::WARNING::2014-10-04 06:26:14,876::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 06:26:14,877::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 06:26:14,877::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 06:26:24,878::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 06:26:24,878::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109198::WARNING::2014-10-04 06:27:24,878::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 06:27:24,878::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 06:27:24,879::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 06:27:34,879::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 06:27:34,880::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 06:27:34,880::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109199::WARNING::2014-10-04 06:28:34,880::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 06:28:34,881::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 06:28:34,881::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 06:28:44,882::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 06:28:44,882::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109200::WARNING::2014-10-04 06:29:44,882::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 06:29:44,883::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 06:29:44,883::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 06:29:54,884::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 06:29:54,884::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109201::WARNING::2014-10-04 06:30:54,884::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 06:30:54,884::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 06:30:54,885::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 06:31:04,886::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 06:31:04,886::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109202::WARNING::2014-10-04 06:32:04,886::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 06:32:04,886::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 06:32:04,886::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 06:32:14,888::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 06:32:14,888::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109203::WARNING::2014-10-04 06:33:14,888::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 06:33:14,888::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 06:33:14,888::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 06:33:24,889::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 06:33:24,890::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 06:33:24,890::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109204::WARNING::2014-10-04 06:34:24,890::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 06:34:24,890::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 06:34:24,890::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 06:34:34,891::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 06:34:34,892::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109205::WARNING::2014-10-04 06:35:34,891::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 06:35:34,892::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 06:35:34,892::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 06:35:44,893::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 06:35:44,893::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109206::WARNING::2014-10-04 06:36:44,893::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 06:36:44,894::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 06:36:44,894::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 06:36:54,895::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 06:36:54,895::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109207::WARNING::2014-10-04 06:37:54,895::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 06:37:54,895::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 06:37:54,896::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 06:38:04,897::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 06:38:04,897::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109208::WARNING::2014-10-04 06:39:04,897::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 06:39:04,897::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 06:39:04,897::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 06:39:14,898::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 06:39:14,899::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 06:39:14,899::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109209::WARNING::2014-10-04 06:40:14,899::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 06:40:14,899::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 06:40:14,899::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 06:40:24,901::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 06:40:24,901::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109210::WARNING::2014-10-04 06:41:24,900::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 06:41:24,901::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 06:41:24,901::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 06:41:34,903::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 06:41:34,903::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109211::WARNING::2014-10-04 06:42:34,902::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 06:42:34,903::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 06:42:34,903::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 06:42:44,904::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 06:42:44,905::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109212::WARNING::2014-10-04 06:43:44,904::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 06:43:44,905::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 06:43:44,905::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 06:43:54,906::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 06:43:54,907::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109213::WARNING::2014-10-04 06:44:54,906::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 06:44:54,907::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 06:44:54,907::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 06:45:04,907::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 06:45:04,908::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 06:45:04,909::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109214::WARNING::2014-10-04 06:46:04,908::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 06:46:04,909::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 06:46:04,909::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 06:46:14,910::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 06:46:14,911::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109215::WARNING::2014-10-04 06:47:14,910::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 06:47:14,911::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 06:47:14,911::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 06:47:24,912::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 06:47:24,913::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109216::WARNING::2014-10-04 06:48:24,912::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 06:48:24,913::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 06:48:24,913::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 06:48:34,914::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 06:48:34,915::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109217::WARNING::2014-10-04 06:49:34,914::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 06:49:34,915::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 06:49:34,915::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 06:49:44,916::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 06:49:44,916::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109218::WARNING::2014-10-04 06:50:44,916::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 06:50:44,917::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 06:50:44,917::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 06:50:54,918::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 06:50:54,919::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 06:50:54,919::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109219::WARNING::2014-10-04 06:51:54,919::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 06:51:54,919::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 06:51:54,919::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 06:52:04,921::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 06:52:04,921::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109220::WARNING::2014-10-04 06:53:04,921::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 06:53:04,921::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 06:53:04,921::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 06:53:14,922::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 06:53:14,923::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109221::WARNING::2014-10-04 06:54:14,922::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 06:54:14,923::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 06:54:14,923::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 06:54:24,924::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 06:54:24,924::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109222::WARNING::2014-10-04 06:55:24,924::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 06:55:24,925::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 06:55:24,925::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 06:55:34,926::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 06:55:34,926::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109223::WARNING::2014-10-04 06:56:34,926::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 06:56:34,926::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 06:56:34,927::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 06:56:35,420::__init__::232::IOProcess::(_processLogs) (92148) Queuing response >Thread-18::DEBUG::2014-10-04 06:56:35,420::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 06:56:35,421::__init__::232::IOProcess::(_processLogs) (92208) Got request for method 'glob' >Thread-48::DEBUG::2014-10-04 06:56:44,927::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 06:56:44,928::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 06:56:44,928::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109224::WARNING::2014-10-04 06:57:44,928::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 06:57:44,928::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 06:57:44,929::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 06:57:54,930::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 06:57:54,930::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109225::WARNING::2014-10-04 06:58:54,930::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 06:58:54,930::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 06:58:54,930::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 06:59:04,932::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 06:59:04,932::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109226::WARNING::2014-10-04 07:00:04,932::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 07:00:04,932::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 07:00:04,932::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 07:00:14,933::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 07:00:14,934::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109227::WARNING::2014-10-04 07:01:14,933::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 07:01:14,934::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 07:01:14,934::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 07:01:24,935::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 07:01:24,935::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109228::WARNING::2014-10-04 07:02:24,935::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 07:02:24,935::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 07:02:24,936::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 07:02:34,936::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 07:02:34,937::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 07:02:34,937::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109229::WARNING::2014-10-04 07:03:34,937::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 07:03:34,937::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 07:03:34,937::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 07:03:44,939::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 07:03:44,939::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109230::WARNING::2014-10-04 07:04:44,939::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 07:04:44,939::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 07:04:44,939::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 07:04:54,940::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 07:04:54,941::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109231::WARNING::2014-10-04 07:05:54,940::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 07:05:54,941::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 07:05:54,941::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 07:06:04,942::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 07:06:04,942::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-04 07:06:39,580::__init__::232::IOProcess::(_processLogs) (92149) Queuing response >Thread-18::DEBUG::2014-10-04 07:06:39,581::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 07:06:39,581::__init__::232::IOProcess::(_processLogs) (92209) Got request for method 'glob' >Thread-109232::WARNING::2014-10-04 07:07:04,942::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 07:07:04,943::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 07:07:04,943::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 07:07:14,944::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 07:07:14,944::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109233::WARNING::2014-10-04 07:08:14,944::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 07:08:14,944::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 07:08:14,945::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 07:08:24,945::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 07:08:24,946::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 07:08:24,946::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109234::WARNING::2014-10-04 07:09:24,946::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 07:09:24,946::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 07:09:24,946::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 07:09:34,948::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 07:09:34,948::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109235::WARNING::2014-10-04 07:10:34,947::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 07:10:34,948::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 07:10:34,948::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 07:10:44,949::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 07:10:44,949::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109236::WARNING::2014-10-04 07:11:44,949::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 07:11:44,950::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 07:11:44,950::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 07:11:54,951::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 07:11:54,951::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109237::WARNING::2014-10-04 07:12:54,951::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 07:12:54,952::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 07:12:54,952::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 07:13:04,953::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 07:13:04,953::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109238::WARNING::2014-10-04 07:14:04,953::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 07:14:04,954::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 07:14:04,954::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 07:14:14,954::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 07:14:14,955::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 07:14:14,955::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109239::WARNING::2014-10-04 07:15:14,955::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 07:15:14,956::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 07:15:14,956::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 07:15:24,957::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 07:15:24,957::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109240::WARNING::2014-10-04 07:16:24,957::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 07:16:24,958::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 07:16:24,958::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 07:16:34,959::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 07:16:34,959::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109241::WARNING::2014-10-04 07:17:34,959::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 07:17:34,960::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 07:17:34,960::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 07:17:44,961::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 07:17:44,961::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109242::WARNING::2014-10-04 07:18:44,961::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 07:18:44,962::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 07:18:44,962::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 07:18:54,963::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 07:18:54,963::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109243::WARNING::2014-10-04 07:19:54,963::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 07:19:54,964::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 07:19:54,964::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 07:20:04,964::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 07:20:04,965::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 07:20:04,965::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109244::WARNING::2014-10-04 07:21:04,965::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 07:21:04,966::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 07:21:04,966::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 07:21:14,967::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 07:21:14,967::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109245::WARNING::2014-10-04 07:22:14,967::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 07:22:14,968::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 07:22:14,968::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 07:22:24,969::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 07:22:24,969::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109246::WARNING::2014-10-04 07:23:24,969::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 07:23:24,970::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 07:23:24,970::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 07:23:34,971::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 07:23:34,971::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109247::WARNING::2014-10-04 07:24:34,971::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 07:24:34,972::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 07:24:34,972::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 07:24:44,973::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 07:24:44,973::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109248::WARNING::2014-10-04 07:25:44,973::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 07:25:44,973::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 07:25:44,974::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 07:25:54,974::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 07:25:54,975::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 07:25:54,975::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109249::WARNING::2014-10-04 07:26:54,975::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 07:26:54,975::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 07:26:54,976::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 07:27:04,977::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 07:27:04,977::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109250::WARNING::2014-10-04 07:28:04,977::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 07:28:04,977::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 07:28:04,977::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 07:28:14,979::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 07:28:14,979::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109251::WARNING::2014-10-04 07:29:14,978::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 07:29:14,979::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 07:29:14,979::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 07:29:24,980::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 07:29:24,981::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109252::WARNING::2014-10-04 07:30:24,980::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 07:30:24,981::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 07:30:24,981::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 07:30:34,982::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 07:30:34,982::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109253::WARNING::2014-10-04 07:31:34,982::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 07:31:34,983::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 07:31:34,983::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 07:31:44,983::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 07:31:44,984::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 07:31:44,984::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109254::WARNING::2014-10-04 07:32:44,984::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 07:32:44,985::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 07:32:44,985::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 07:32:54,986::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 07:32:54,986::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109255::WARNING::2014-10-04 07:33:54,986::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 07:33:54,987::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 07:33:54,987::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 07:34:04,988::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 07:34:04,989::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109256::WARNING::2014-10-04 07:35:04,988::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 07:35:04,989::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 07:35:04,989::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 07:35:14,990::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 07:35:14,990::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109257::WARNING::2014-10-04 07:36:14,990::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 07:36:14,991::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 07:36:14,991::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 07:36:24,992::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 07:36:24,992::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109258::WARNING::2014-10-04 07:37:24,992::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 07:37:24,993::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 07:37:24,993::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 07:37:34,993::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 07:37:34,994::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 07:37:34,994::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109259::WARNING::2014-10-04 07:38:34,994::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 07:38:34,995::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 07:38:34,995::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 07:38:44,996::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 07:38:44,996::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109260::WARNING::2014-10-04 07:39:44,996::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 07:39:44,996::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 07:39:44,997::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 07:39:54,998::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 07:39:54,998::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109261::WARNING::2014-10-04 07:40:54,998::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 07:40:54,998::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 07:40:54,999::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 07:41:05,000::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 07:41:05,000::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109262::WARNING::2014-10-04 07:42:05,000::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 07:42:05,000::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 07:42:05,000::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 07:42:15,002::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 07:42:15,002::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109263::WARNING::2014-10-04 07:43:15,002::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 07:43:15,002::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 07:43:15,002::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 07:43:25,003::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 07:43:25,004::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 07:43:25,004::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109264::WARNING::2014-10-04 07:44:25,004::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 07:44:25,004::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 07:44:25,005::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 07:44:35,006::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 07:44:35,006::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109265::WARNING::2014-10-04 07:45:35,006::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 07:45:35,006::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 07:45:35,007::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 07:45:45,008::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 07:45:45,008::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109266::WARNING::2014-10-04 07:46:45,008::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 07:46:45,008::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 07:46:45,008::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 07:46:55,010::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 07:46:55,010::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109267::WARNING::2014-10-04 07:47:55,010::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 07:47:55,010::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 07:47:55,010::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 07:48:05,012::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 07:48:05,012::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109268::WARNING::2014-10-04 07:49:05,012::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 07:49:05,012::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 07:49:05,012::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 07:49:15,013::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 07:49:15,014::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 07:49:15,014::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109269::WARNING::2014-10-04 07:50:15,014::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 07:50:15,014::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 07:50:15,014::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 07:50:25,016::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 07:50:25,016::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109270::WARNING::2014-10-04 07:51:25,016::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 07:51:25,016::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 07:51:25,016::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 07:51:35,018::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 07:51:35,018::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109271::WARNING::2014-10-04 07:52:35,018::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 07:52:35,018::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 07:52:35,018::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 07:52:45,020::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 07:52:45,020::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109272::WARNING::2014-10-04 07:53:45,020::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 07:53:45,020::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 07:53:45,020::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 07:53:55,021::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 07:53:55,021::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109273::WARNING::2014-10-04 07:54:55,021::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 07:54:55,022::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 07:54:55,022::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 07:55:05,022::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 07:55:05,023::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 07:55:05,023::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109274::WARNING::2014-10-04 07:56:05,023::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 07:56:05,024::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 07:56:05,024::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 07:56:15,025::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 07:56:15,025::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109275::WARNING::2014-10-04 07:57:15,025::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 07:57:15,025::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 07:57:15,026::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 07:57:25,027::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 07:57:25,027::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109276::WARNING::2014-10-04 07:58:25,027::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 07:58:25,027::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 07:58:25,027::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 07:58:35,029::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 07:58:35,029::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109277::WARNING::2014-10-04 07:59:35,028::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 07:59:35,029::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 07:59:35,029::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 07:59:45,030::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 07:59:45,030::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109278::WARNING::2014-10-04 08:00:45,030::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 08:00:45,031::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 08:00:45,031::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 08:00:55,031::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 08:00:55,032::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 08:00:55,032::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109279::WARNING::2014-10-04 08:01:55,032::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 08:01:55,033::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 08:01:55,033::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 08:02:05,034::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 08:02:05,034::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109280::WARNING::2014-10-04 08:03:05,034::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 08:03:05,034::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 08:03:05,035::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 08:03:15,036::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 08:03:15,036::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109281::WARNING::2014-10-04 08:04:15,036::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 08:04:15,036::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 08:04:15,036::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 08:04:25,038::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 08:04:25,038::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109282::WARNING::2014-10-04 08:05:25,037::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 08:05:25,038::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 08:05:25,038::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 08:05:35,039::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 08:05:35,040::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109283::WARNING::2014-10-04 08:06:35,039::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 08:06:35,040::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 08:06:35,040::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 08:06:45,040::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 08:06:45,041::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 08:06:45,041::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109284::WARNING::2014-10-04 08:07:45,041::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 08:07:45,042::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 08:07:45,042::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 08:07:55,043::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 08:07:55,043::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109285::WARNING::2014-10-04 08:08:55,043::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 08:08:55,044::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 08:08:55,044::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 08:09:05,045::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 08:09:05,045::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109286::WARNING::2014-10-04 08:10:05,045::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 08:10:05,045::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 08:10:05,045::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 08:10:15,047::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 08:10:15,047::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109287::WARNING::2014-10-04 08:11:15,047::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 08:11:15,047::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 08:11:15,047::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 08:11:25,048::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 08:11:25,049::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109288::WARNING::2014-10-04 08:12:25,048::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 08:12:25,049::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 08:12:25,049::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 08:12:35,049::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 08:12:35,051::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 08:12:35,051::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109289::WARNING::2014-10-04 08:13:35,050::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 08:13:35,051::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 08:13:35,051::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 08:13:45,052::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 08:13:45,053::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109290::WARNING::2014-10-04 08:14:45,052::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 08:14:45,053::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 08:14:45,053::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 08:14:55,054::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 08:14:55,055::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109291::WARNING::2014-10-04 08:15:55,054::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 08:15:55,055::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 08:15:55,055::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 08:16:05,056::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 08:16:05,057::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109292::WARNING::2014-10-04 08:17:05,056::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 08:17:05,057::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 08:17:05,057::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 08:17:15,058::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 08:17:15,059::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109293::WARNING::2014-10-04 08:18:15,058::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 08:18:15,059::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 08:18:15,059::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 08:18:25,060::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 08:18:25,061::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 08:18:25,061::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109294::WARNING::2014-10-04 08:19:25,061::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 08:19:25,061::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 08:19:25,061::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 08:19:35,063::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 08:19:35,063::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109295::WARNING::2014-10-04 08:20:35,063::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 08:20:35,063::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 08:20:35,063::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 08:20:45,064::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 08:20:45,065::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109296::WARNING::2014-10-04 08:21:45,064::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 08:21:45,065::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 08:21:45,065::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 08:21:55,066::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 08:21:55,067::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109297::WARNING::2014-10-04 08:22:55,066::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 08:22:55,067::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 08:22:55,067::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 08:23:05,068::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 08:23:05,068::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109298::WARNING::2014-10-04 08:24:05,068::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 08:24:05,069::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 08:24:05,069::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 08:24:15,069::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 08:24:15,070::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 08:24:15,070::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109299::WARNING::2014-10-04 08:25:15,070::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 08:25:15,071::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 08:25:15,071::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 08:25:25,072::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 08:25:25,072::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109300::WARNING::2014-10-04 08:26:25,072::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 08:26:25,073::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 08:26:25,073::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 08:26:35,074::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 08:26:35,074::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-04 08:27:12,860::__init__::232::IOProcess::(_processLogs) (92150) Queuing response >Thread-18::DEBUG::2014-10-04 08:27:12,861::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 08:27:12,861::__init__::232::IOProcess::(_processLogs) (92210) Got request for method 'glob' >Thread-109301::WARNING::2014-10-04 08:27:35,074::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 08:27:35,074::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 08:27:35,075::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 08:27:45,076::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 08:27:45,076::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109302::WARNING::2014-10-04 08:28:45,076::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 08:28:45,076::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 08:28:45,077::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 08:28:55,078::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 08:28:55,078::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109303::WARNING::2014-10-04 08:29:55,078::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 08:29:55,078::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 08:29:55,078::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 08:30:05,079::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 08:30:05,080::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 08:30:05,080::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109304::WARNING::2014-10-04 08:31:05,080::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 08:31:05,080::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 08:31:05,081::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 08:31:15,082::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 08:31:15,082::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109305::WARNING::2014-10-04 08:32:15,082::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 08:32:15,082::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 08:32:15,082::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 08:32:25,084::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 08:32:25,084::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109306::WARNING::2014-10-04 08:33:25,084::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 08:33:25,084::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 08:33:25,084::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 08:33:35,085::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 08:33:35,086::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109307::WARNING::2014-10-04 08:34:35,085::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 08:34:35,086::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 08:34:35,086::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 08:34:45,087::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 08:34:45,087::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109308::WARNING::2014-10-04 08:35:45,087::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 08:35:45,088::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 08:35:45,088::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 08:35:55,088::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 08:35:55,089::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 08:35:55,089::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109309::WARNING::2014-10-04 08:36:55,089::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 08:36:55,090::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 08:36:55,090::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 08:37:05,091::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 08:37:05,091::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-04 08:37:17,020::__init__::232::IOProcess::(_processLogs) (92151) Queuing response >Thread-18::DEBUG::2014-10-04 08:37:17,021::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 08:37:17,021::__init__::232::IOProcess::(_processLogs) (92211) Got request for method 'glob' >Thread-18::DEBUG::2014-10-04 08:37:17,021::__init__::232::IOProcess::(_processLogs) (92152) Queuing response >Thread-18::DEBUG::2014-10-04 08:37:17,021::__init__::232::IOProcess::(_processLogs) (92153) Queuing response >Thread-18::DEBUG::2014-10-04 08:37:17,021::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 08:37:17,021::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 08:37:17,021::__init__::232::IOProcess::(_processLogs) (92213) Got request for method 'glob' >Thread-18::DEBUG::2014-10-04 08:37:17,021::__init__::232::IOProcess::(_processLogs) (92212) Got request for method 'glob' >Thread-109310::WARNING::2014-10-04 08:38:05,091::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 08:38:05,092::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 08:38:05,092::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 08:38:15,093::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 08:38:15,093::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109311::WARNING::2014-10-04 08:39:15,093::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 08:39:15,093::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 08:39:15,094::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 08:39:25,095::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 08:39:25,095::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109312::WARNING::2014-10-04 08:40:25,095::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 08:40:25,095::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 08:40:25,096::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 08:40:35,097::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 08:40:35,097::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109313::WARNING::2014-10-04 08:41:35,097::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 08:41:35,097::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 08:41:35,098::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 08:41:45,098::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 08:41:45,099::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 08:41:45,099::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109314::WARNING::2014-10-04 08:42:45,099::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 08:42:45,099::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 08:42:45,100::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 08:42:55,101::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 08:42:55,101::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109315::WARNING::2014-10-04 08:43:55,101::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 08:43:55,101::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 08:43:55,101::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 08:44:05,103::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 08:44:05,103::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109316::WARNING::2014-10-04 08:45:05,103::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 08:45:05,103::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 08:45:05,103::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 08:45:15,104::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 08:45:15,105::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109317::WARNING::2014-10-04 08:46:15,104::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 08:46:15,105::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 08:46:15,105::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 08:46:25,106::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 08:46:25,107::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-04 08:47:21,180::__init__::232::IOProcess::(_processLogs) (92154) Queuing response >Thread-18::DEBUG::2014-10-04 08:47:21,181::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 08:47:21,181::__init__::232::IOProcess::(_processLogs) (92214) Got request for method 'glob' >Thread-18::DEBUG::2014-10-04 08:47:21,181::__init__::232::IOProcess::(_processLogs) (92155) Queuing response >Thread-18::DEBUG::2014-10-04 08:47:21,181::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 08:47:21,181::__init__::232::IOProcess::(_processLogs) (92215) Got request for method 'glob' >Thread-109318::WARNING::2014-10-04 08:47:25,106::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 08:47:25,107::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 08:47:25,107::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 08:47:35,108::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 08:47:35,109::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 08:47:35,109::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109319::WARNING::2014-10-04 08:48:35,109::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 08:48:35,109::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 08:48:35,109::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 08:48:45,111::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 08:48:45,111::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109320::WARNING::2014-10-04 08:49:45,110::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 08:49:45,111::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 08:49:45,111::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 08:49:55,112::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 08:49:55,113::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109321::WARNING::2014-10-04 08:50:55,112::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 08:50:55,113::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 08:50:55,113::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 08:51:05,115::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 08:51:05,115::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109322::WARNING::2014-10-04 08:52:05,114::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 08:52:05,115::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 08:52:05,115::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 08:52:15,116::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 08:52:15,117::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109323::WARNING::2014-10-04 08:53:15,116::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 08:53:15,117::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 08:53:15,117::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 08:53:25,117::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 08:53:25,118::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 08:53:25,118::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109324::WARNING::2014-10-04 08:54:25,118::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 08:54:25,119::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 08:54:25,119::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 08:54:35,120::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 08:54:35,120::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109325::WARNING::2014-10-04 08:55:35,120::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 08:55:35,120::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 08:55:35,120::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 08:55:45,122::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 08:55:45,122::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109326::WARNING::2014-10-04 08:56:45,122::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 08:56:45,122::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 08:56:45,122::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 08:56:55,123::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 08:56:55,124::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-04 08:57:25,340::__init__::232::IOProcess::(_processLogs) (92156) Queuing response >Thread-18::DEBUG::2014-10-04 08:57:25,341::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 08:57:25,341::__init__::232::IOProcess::(_processLogs) (92216) Got request for method 'glob' >Thread-18::DEBUG::2014-10-04 08:57:25,341::__init__::232::IOProcess::(_processLogs) (92157) Queuing response >Thread-18::DEBUG::2014-10-04 08:57:25,341::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 08:57:25,341::__init__::232::IOProcess::(_processLogs) (92217) Got request for method 'glob' >Thread-109327::WARNING::2014-10-04 08:57:55,123::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 08:57:55,124::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 08:57:55,124::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 08:58:05,125::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 08:58:05,125::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109328::WARNING::2014-10-04 08:59:05,125::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 08:59:05,125::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 08:59:05,126::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 08:59:15,126::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 08:59:15,127::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 08:59:15,127::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109329::WARNING::2014-10-04 09:00:15,127::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 09:00:15,127::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 09:00:15,127::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 09:00:25,129::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 09:00:25,129::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109330::WARNING::2014-10-04 09:01:25,129::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 09:01:25,129::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 09:01:25,129::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 09:01:35,130::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 09:01:35,131::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109331::WARNING::2014-10-04 09:02:35,130::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 09:02:35,131::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 09:02:35,131::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 09:02:45,132::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 09:02:45,133::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109332::WARNING::2014-10-04 09:03:45,132::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 09:03:45,133::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 09:03:45,133::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 09:03:55,134::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 09:03:55,134::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109333::WARNING::2014-10-04 09:04:55,134::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 09:04:55,135::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 09:04:55,135::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 09:05:05,135::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 09:05:05,136::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 09:05:05,136::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109334::WARNING::2014-10-04 09:06:05,136::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 09:06:05,137::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 09:06:05,137::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 09:06:15,138::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 09:06:15,138::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109335::WARNING::2014-10-04 09:07:15,138::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 09:07:15,139::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 09:07:15,139::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 09:07:25,140::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 09:07:25,140::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-04 09:07:29,500::__init__::232::IOProcess::(_processLogs) (92158) Queuing response >Thread-18::DEBUG::2014-10-04 09:07:29,501::__init__::232::IOProcess::(_processLogs) (92159) Queuing response >Thread-18::DEBUG::2014-10-04 09:07:29,501::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 09:07:29,501::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 09:07:29,501::__init__::232::IOProcess::(_processLogs) (92219) Got request for method 'glob' >Thread-18::DEBUG::2014-10-04 09:07:29,501::__init__::232::IOProcess::(_processLogs) (92218) Got request for method 'glob' >Thread-109336::WARNING::2014-10-04 09:08:25,140::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 09:08:25,140::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 09:08:25,141::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 09:08:35,142::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 09:08:35,142::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109337::WARNING::2014-10-04 09:09:35,142::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 09:09:35,142::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 09:09:35,142::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 09:09:45,144::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 09:09:45,144::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109338::WARNING::2014-10-04 09:10:45,143::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 09:10:45,144::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 09:10:45,144::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 09:10:55,145::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 09:10:55,146::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 09:10:55,146::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109339::WARNING::2014-10-04 09:11:55,146::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 09:11:55,146::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 09:11:55,146::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 09:12:05,148::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 09:12:05,148::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109340::WARNING::2014-10-04 09:13:05,147::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 09:13:05,148::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 09:13:05,148::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 09:13:15,149::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 09:13:15,150::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109341::WARNING::2014-10-04 09:14:15,149::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 09:14:15,150::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 09:14:15,150::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 09:14:25,151::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 09:14:25,151::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109342::WARNING::2014-10-04 09:15:25,151::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 09:15:25,152::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 09:15:25,152::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 09:15:35,154::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 09:15:35,154::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109343::WARNING::2014-10-04 09:16:35,154::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 09:16:35,154::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 09:16:35,154::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 09:16:45,155::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 09:16:45,156::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 09:16:45,156::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-04 09:17:35,708::__init__::232::IOProcess::(_processLogs) (92160) Queuing response >Thread-18::DEBUG::2014-10-04 09:17:35,709::__init__::232::IOProcess::(_processLogs) (92161) Queuing response >Thread-18::DEBUG::2014-10-04 09:17:35,709::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 09:17:35,709::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 09:17:35,709::__init__::232::IOProcess::(_processLogs) (92220) Got request for method 'glob' >Thread-18::DEBUG::2014-10-04 09:17:35,709::__init__::232::IOProcess::(_processLogs) (92221) Got request for method 'glob' >Thread-18::DEBUG::2014-10-04 09:17:35,709::__init__::232::IOProcess::(_processLogs) (92162) Queuing response >Thread-18::DEBUG::2014-10-04 09:17:35,709::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 09:17:35,710::__init__::232::IOProcess::(_processLogs) (92222) Got request for method 'glob' >Thread-109344::WARNING::2014-10-04 09:17:45,156::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 09:17:45,156::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 09:17:45,157::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 09:17:55,158::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 09:17:55,158::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109345::WARNING::2014-10-04 09:18:55,158::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 09:18:55,158::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 09:18:55,158::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 09:19:05,160::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 09:19:05,160::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109346::WARNING::2014-10-04 09:20:05,160::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 09:20:05,160::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 09:20:05,160::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 09:20:15,162::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 09:20:15,162::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109347::WARNING::2014-10-04 09:21:15,162::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 09:21:15,162::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 09:21:15,162::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 09:21:25,164::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 09:21:25,164::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109348::WARNING::2014-10-04 09:22:25,164::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 09:22:25,164::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 09:22:25,164::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 09:22:35,165::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 09:22:35,166::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 09:22:35,166::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109349::WARNING::2014-10-04 09:23:35,166::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 09:23:35,166::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 09:23:35,166::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 09:23:45,168::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 09:23:45,168::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109350::WARNING::2014-10-04 09:24:45,167::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 09:24:45,168::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 09:24:45,168::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 09:24:55,169::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 09:24:55,170::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109351::WARNING::2014-10-04 09:25:55,169::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 09:25:55,170::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 09:25:55,170::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 09:26:05,171::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 09:26:05,171::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109352::WARNING::2014-10-04 09:27:05,171::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 09:27:05,172::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 09:27:05,172::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 09:27:15,173::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 09:27:15,173::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-04 09:27:39,868::__init__::232::IOProcess::(_processLogs) (92163) Queuing response >Thread-18::DEBUG::2014-10-04 09:27:39,869::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 09:27:39,869::__init__::232::IOProcess::(_processLogs) (92223) Got request for method 'glob' >Thread-18::DEBUG::2014-10-04 09:27:39,869::__init__::232::IOProcess::(_processLogs) (92164) Queuing response >Thread-18::DEBUG::2014-10-04 09:27:39,869::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 09:27:39,869::__init__::232::IOProcess::(_processLogs) (92224) Got request for method 'glob' >Thread-109353::WARNING::2014-10-04 09:28:15,173::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 09:28:15,173::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 09:28:15,174::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 09:28:25,174::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 09:28:25,175::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 09:28:25,175::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109354::WARNING::2014-10-04 09:29:25,175::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 09:29:25,175::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 09:29:25,176::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 09:29:35,177::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 09:29:35,177::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109355::WARNING::2014-10-04 09:30:35,177::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 09:30:35,177::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 09:30:35,177::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 09:30:45,179::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 09:30:45,179::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109356::WARNING::2014-10-04 09:31:45,179::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 09:31:45,179::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 09:31:45,179::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 09:31:55,180::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 09:31:55,181::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109357::WARNING::2014-10-04 09:32:55,180::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 09:32:55,181::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 09:32:55,181::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 09:33:05,182::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 09:33:05,182::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109358::WARNING::2014-10-04 09:34:05,182::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 09:34:05,183::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 09:34:05,183::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 09:34:15,183::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 09:34:15,184::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 09:34:15,185::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109359::WARNING::2014-10-04 09:35:15,184::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 09:35:15,185::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 09:35:15,185::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 09:35:25,186::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 09:35:25,186::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109360::WARNING::2014-10-04 09:36:25,186::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 09:36:25,187::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 09:36:25,187::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 09:36:35,188::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 09:36:35,188::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109361::WARNING::2014-10-04 09:37:35,188::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 09:37:35,189::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 09:37:35,189::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 09:37:45,190::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 09:37:45,190::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-04 09:37:46,076::__init__::232::IOProcess::(_processLogs) (92165) Queuing response >Thread-18::DEBUG::2014-10-04 09:37:46,076::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 09:37:46,077::__init__::232::IOProcess::(_processLogs) (92225) Got request for method 'glob' >Thread-18::DEBUG::2014-10-04 09:37:46,077::__init__::232::IOProcess::(_processLogs) (92166) Queuing response >Thread-18::DEBUG::2014-10-04 09:37:46,077::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 09:37:46,077::__init__::232::IOProcess::(_processLogs) (92226) Got request for method 'glob' >Thread-109362::WARNING::2014-10-04 09:38:45,190::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 09:38:45,190::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 09:38:45,191::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 09:38:55,192::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 09:38:55,192::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109363::WARNING::2014-10-04 09:39:55,192::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 09:39:55,192::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 09:39:55,192::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 09:40:05,193::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 09:40:05,194::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 09:40:05,194::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109364::WARNING::2014-10-04 09:41:05,194::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 09:41:05,194::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 09:41:05,194::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 09:41:15,196::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 09:41:15,196::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109365::WARNING::2014-10-04 09:42:15,195::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 09:42:15,196::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 09:42:15,196::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 09:42:25,197::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 09:42:25,198::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109366::WARNING::2014-10-04 09:43:25,197::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 09:43:25,198::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 09:43:25,198::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 09:43:35,199::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 09:43:35,200::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109367::WARNING::2014-10-04 09:44:35,199::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 09:44:35,200::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 09:44:35,200::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 09:44:45,201::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 09:44:45,201::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109368::WARNING::2014-10-04 09:45:45,201::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 09:45:45,202::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 09:45:45,202::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 09:45:55,202::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 09:45:55,203::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 09:45:55,204::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109369::WARNING::2014-10-04 09:46:55,203::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 09:46:55,204::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 09:46:55,204::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 09:47:05,205::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 09:47:05,205::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-04 09:47:50,236::__init__::232::IOProcess::(_processLogs) (92167) Queuing response >Thread-18::DEBUG::2014-10-04 09:47:50,237::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 09:47:50,237::__init__::232::IOProcess::(_processLogs) (92227) Got request for method 'glob' >Thread-109370::WARNING::2014-10-04 09:48:05,205::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 09:48:05,206::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 09:48:05,206::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 09:48:15,207::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 09:48:15,207::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109371::WARNING::2014-10-04 09:49:15,207::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 09:49:15,208::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 09:49:15,208::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 09:49:25,209::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 09:49:25,209::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109372::WARNING::2014-10-04 09:50:25,209::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 09:50:25,210::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 09:50:25,210::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 09:50:35,211::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 09:50:35,211::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109373::WARNING::2014-10-04 09:51:35,211::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 09:51:35,212::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 09:51:35,212::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 09:51:45,212::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 09:51:45,214::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 09:51:45,214::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109374::WARNING::2014-10-04 09:52:45,214::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 09:52:45,215::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 09:52:45,215::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 09:52:55,216::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 09:52:55,216::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109375::WARNING::2014-10-04 09:53:55,216::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 09:53:55,217::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 09:53:55,217::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 09:54:05,218::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 09:54:05,218::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109376::WARNING::2014-10-04 09:55:05,218::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 09:55:05,218::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 09:55:05,219::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 09:55:15,220::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 09:55:15,220::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109377::WARNING::2014-10-04 09:56:15,220::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 09:56:15,220::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 09:56:15,220::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 09:56:25,221::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 09:56:25,222::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109378::WARNING::2014-10-04 09:57:25,221::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 09:57:25,222::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 09:57:25,222::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 09:57:35,222::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 09:57:35,223::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 09:57:35,223::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-04 09:57:54,396::__init__::232::IOProcess::(_processLogs) (92169) Queuing response >Thread-18::DEBUG::2014-10-04 09:57:54,397::__init__::232::IOProcess::(_processLogs) (92168) Queuing response >Thread-18::DEBUG::2014-10-04 09:57:54,397::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 09:57:54,397::__init__::232::IOProcess::(_processLogs) (92228) Got request for method 'glob' >Thread-18::DEBUG::2014-10-04 09:57:54,397::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 09:57:54,397::__init__::232::IOProcess::(_processLogs) (92229) Got request for method 'glob' >Thread-109379::WARNING::2014-10-04 09:58:35,223::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 09:58:35,224::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 09:58:35,224::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 09:58:45,225::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 09:58:45,225::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109380::WARNING::2014-10-04 09:59:45,225::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 09:59:45,225::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 09:59:45,226::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 09:59:55,227::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 09:59:55,227::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109381::WARNING::2014-10-04 10:00:55,227::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 10:00:55,227::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 10:00:55,227::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 10:01:05,229::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 10:01:05,229::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109382::WARNING::2014-10-04 10:02:05,229::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 10:02:05,229::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 10:02:05,229::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 10:02:15,231::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 10:02:15,231::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109383::WARNING::2014-10-04 10:03:15,231::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 10:03:15,231::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 10:03:15,231::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 10:03:25,232::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 10:03:25,233::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 10:03:25,233::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109384::WARNING::2014-10-04 10:04:25,233::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 10:04:25,233::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 10:04:25,233::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 10:04:35,235::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 10:04:35,235::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109385::WARNING::2014-10-04 10:05:35,235::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 10:05:35,235::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 10:05:35,235::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 10:05:45,236::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 10:05:45,237::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109386::WARNING::2014-10-04 10:06:45,236::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 10:06:45,237::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 10:06:45,237::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 10:06:55,238::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 10:06:55,239::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109387::WARNING::2014-10-04 10:07:55,238::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 10:07:55,239::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 10:07:55,239::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 10:07:58,556::__init__::232::IOProcess::(_processLogs) (92171) Queuing response >Thread-18::DEBUG::2014-10-04 10:07:58,557::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 10:07:58,557::__init__::232::IOProcess::(_processLogs) (92230) Got request for method 'glob' >Thread-18::DEBUG::2014-10-04 10:07:58,557::__init__::232::IOProcess::(_processLogs) (92170) Queuing response >Thread-18::DEBUG::2014-10-04 10:07:58,557::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 10:07:58,557::__init__::232::IOProcess::(_processLogs) (92231) Got request for method 'glob' >Thread-18::DEBUG::2014-10-04 10:08:05,240::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 10:08:05,240::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109388::WARNING::2014-10-04 10:09:05,240::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 10:09:05,241::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 10:09:05,241::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 10:09:15,241::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 10:09:15,242::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 10:09:15,242::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109389::WARNING::2014-10-04 10:10:15,242::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 10:10:15,243::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 10:10:15,243::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 10:10:25,244::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 10:10:25,244::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109390::WARNING::2014-10-04 10:11:25,244::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 10:11:25,245::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 10:11:25,245::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 10:11:35,246::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 10:11:35,246::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109391::WARNING::2014-10-04 10:12:35,246::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 10:12:35,246::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 10:12:35,247::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 10:12:45,248::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 10:12:45,248::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109392::WARNING::2014-10-04 10:13:45,248::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 10:13:45,248::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 10:13:45,248::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 10:13:55,250::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 10:13:55,250::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109393::WARNING::2014-10-04 10:14:55,250::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 10:14:55,250::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 10:14:55,250::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 10:15:05,251::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 10:15:05,252::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 10:15:05,252::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109394::WARNING::2014-10-04 10:16:05,252::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 10:16:05,252::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 10:16:05,252::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 10:16:15,254::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 10:16:15,254::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109395::WARNING::2014-10-04 10:17:15,253::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 10:17:15,254::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 10:17:15,254::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 10:17:25,255::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 10:17:25,256::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-04 10:18:02,716::__init__::232::IOProcess::(_processLogs) (92172) Queuing response >Thread-18::DEBUG::2014-10-04 10:18:02,717::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 10:18:02,717::__init__::232::IOProcess::(_processLogs) (92232) Got request for method 'glob' >Thread-18::DEBUG::2014-10-04 10:18:02,717::__init__::232::IOProcess::(_processLogs) (92173) Queuing response >Thread-18::DEBUG::2014-10-04 10:18:02,717::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 10:18:02,717::__init__::232::IOProcess::(_processLogs) (92233) Got request for method 'glob' >Thread-109396::WARNING::2014-10-04 10:18:25,255::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 10:18:25,256::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 10:18:25,256::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 10:18:35,257::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 10:18:35,257::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109397::WARNING::2014-10-04 10:19:35,257::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 10:19:35,258::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 10:19:35,258::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 10:19:45,260::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 10:19:45,260::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109398::WARNING::2014-10-04 10:20:45,259::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 10:20:45,260::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 10:20:45,260::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 10:20:55,261::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 10:20:55,262::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 10:20:55,262::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109399::WARNING::2014-10-04 10:21:55,262::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 10:21:55,262::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 10:21:55,262::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 10:22:05,264::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 10:22:05,264::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109400::WARNING::2014-10-04 10:23:05,264::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 10:23:05,264::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 10:23:05,264::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 10:23:15,265::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 10:23:15,266::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109401::WARNING::2014-10-04 10:24:15,265::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 10:24:15,266::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 10:24:15,266::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 10:24:25,267::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 10:24:25,267::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109402::WARNING::2014-10-04 10:25:25,267::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 10:25:25,268::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 10:25:25,268::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 10:25:35,269::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 10:25:35,269::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109403::WARNING::2014-10-04 10:26:35,269::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 10:26:35,270::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 10:26:35,270::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 10:26:45,270::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 10:26:45,271::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 10:26:45,271::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109404::WARNING::2014-10-04 10:27:45,271::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 10:27:45,272::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 10:27:45,272::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 10:27:55,273::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 10:27:55,273::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-04 10:28:08,924::__init__::232::IOProcess::(_processLogs) (92174) Queuing response >Thread-18::DEBUG::2014-10-04 10:28:08,925::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 10:28:08,925::__init__::232::IOProcess::(_processLogs) (92234) Got request for method 'glob' >Thread-18::DEBUG::2014-10-04 10:28:08,925::__init__::232::IOProcess::(_processLogs) (92175) Queuing response >Thread-18::DEBUG::2014-10-04 10:28:08,925::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 10:28:08,925::__init__::232::IOProcess::(_processLogs) (92235) Got request for method 'glob' >Thread-109405::WARNING::2014-10-04 10:28:55,273::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 10:28:55,273::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 10:28:55,274::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 10:29:05,275::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 10:29:05,275::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109406::WARNING::2014-10-04 10:30:05,275::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 10:30:05,275::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 10:30:05,276::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 10:30:15,277::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 10:30:15,277::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109407::WARNING::2014-10-04 10:31:15,277::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 10:31:15,277::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 10:31:15,277::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 10:31:25,278::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 10:31:25,279::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109408::WARNING::2014-10-04 10:32:25,278::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 10:32:25,279::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 10:32:25,279::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 10:32:35,279::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 10:32:35,280::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 10:32:35,281::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109409::WARNING::2014-10-04 10:33:35,280::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 10:33:35,281::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 10:33:35,281::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 10:33:45,282::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 10:33:45,282::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109410::WARNING::2014-10-04 10:34:45,282::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 10:34:45,283::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 10:34:45,283::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 10:34:55,284::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 10:34:55,284::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109411::WARNING::2014-10-04 10:35:55,284::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 10:35:55,284::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 10:35:55,285::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 10:36:05,286::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 10:36:05,286::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109412::WARNING::2014-10-04 10:37:05,286::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 10:37:05,286::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 10:37:05,287::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 10:37:15,288::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 10:37:15,288::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-04 10:38:13,084::__init__::232::IOProcess::(_processLogs) (92176) Queuing response >Thread-18::DEBUG::2014-10-04 10:38:13,085::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 10:38:13,085::__init__::232::IOProcess::(_processLogs) (92236) Got request for method 'glob' >Thread-109413::WARNING::2014-10-04 10:38:15,288::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 10:38:15,288::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 10:38:15,288::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 10:38:25,289::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 10:38:25,290::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 10:38:25,290::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109414::WARNING::2014-10-04 10:39:25,290::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 10:39:25,290::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 10:39:25,290::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 10:39:35,291::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 10:39:35,292::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109415::WARNING::2014-10-04 10:40:35,291::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 10:40:35,292::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 10:40:35,292::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 10:40:45,293::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 10:40:45,293::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109416::WARNING::2014-10-04 10:41:45,293::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 10:41:45,294::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 10:41:45,294::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 10:41:55,295::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 10:41:55,295::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109417::WARNING::2014-10-04 10:42:55,295::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 10:42:55,296::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 10:42:55,296::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 10:43:05,297::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 10:43:05,297::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109418::WARNING::2014-10-04 10:44:05,297::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 10:44:05,297::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 10:44:05,298::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 10:44:15,298::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 10:44:15,299::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 10:44:15,299::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109419::WARNING::2014-10-04 10:45:15,299::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 10:45:15,299::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 10:45:15,300::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 10:45:25,301::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 10:45:25,301::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109420::WARNING::2014-10-04 10:46:25,301::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 10:46:25,301::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 10:46:25,302::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 10:46:35,303::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 10:46:35,303::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109421::WARNING::2014-10-04 10:47:35,303::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 10:47:35,303::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 10:47:35,303::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 10:47:45,305::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 10:47:45,305::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-04 10:48:17,244::__init__::232::IOProcess::(_processLogs) (92177) Queuing response >Thread-18::DEBUG::2014-10-04 10:48:17,245::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 10:48:17,245::__init__::232::IOProcess::(_processLogs) (92237) Got request for method 'glob' >Thread-109422::WARNING::2014-10-04 10:48:45,305::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 10:48:45,305::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 10:48:45,305::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 10:48:55,307::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 10:48:55,307::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109423::WARNING::2014-10-04 10:49:55,307::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 10:49:55,307::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 10:49:55,307::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 10:50:05,308::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 10:50:05,309::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 10:50:05,309::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109424::WARNING::2014-10-04 10:51:05,309::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 10:51:05,309::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 10:51:05,309::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 10:51:15,311::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 10:51:15,311::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109425::WARNING::2014-10-04 10:52:15,311::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 10:52:15,311::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 10:52:15,311::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 10:52:25,313::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 10:52:25,313::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109426::WARNING::2014-10-04 10:53:25,313::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 10:53:25,313::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 10:53:25,313::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 10:53:35,315::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 10:53:35,315::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109427::WARNING::2014-10-04 10:54:35,314::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 10:54:35,315::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 10:54:35,315::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 10:54:45,317::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 10:54:45,317::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109428::WARNING::2014-10-04 10:55:45,317::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 10:55:45,317::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 10:55:45,317::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 10:55:55,318::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 10:55:55,319::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 10:55:55,319::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109429::WARNING::2014-10-04 10:56:55,319::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 10:56:55,319::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 10:56:55,319::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 10:57:05,320::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 10:57:05,321::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109430::WARNING::2014-10-04 10:58:05,320::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 10:58:05,321::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 10:58:05,321::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 10:58:15,322::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 10:58:15,322::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-04 10:58:21,404::__init__::232::IOProcess::(_processLogs) (92178) Queuing response >Thread-18::DEBUG::2014-10-04 10:58:21,405::__init__::232::IOProcess::(_processLogs) (92179) Queuing response >Thread-18::DEBUG::2014-10-04 10:58:21,405::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 10:58:21,405::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 10:58:21,405::__init__::232::IOProcess::(_processLogs) (92238) Got request for method 'glob' >Thread-18::DEBUG::2014-10-04 10:58:21,405::__init__::232::IOProcess::(_processLogs) (92239) Got request for method 'glob' >Thread-109431::WARNING::2014-10-04 10:59:15,322::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 10:59:15,323::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 10:59:15,323::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 10:59:25,324::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 10:59:25,324::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109432::WARNING::2014-10-04 11:00:25,324::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 11:00:25,325::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 11:00:25,325::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 11:00:35,326::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 11:00:35,326::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109433::WARNING::2014-10-04 11:01:35,326::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 11:01:35,327::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 11:01:35,327::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 11:01:45,327::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 11:01:45,328::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 11:01:45,328::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109434::WARNING::2014-10-04 11:02:45,328::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 11:02:45,329::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 11:02:45,329::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 11:02:55,330::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 11:02:55,330::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109435::WARNING::2014-10-04 11:03:55,330::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 11:03:55,330::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 11:03:55,331::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 11:04:05,332::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 11:04:05,332::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109436::WARNING::2014-10-04 11:05:05,332::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 11:05:05,332::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 11:05:05,333::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 11:05:15,334::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 11:05:15,334::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109437::WARNING::2014-10-04 11:06:15,334::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 11:06:15,334::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 11:06:15,334::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 11:06:25,336::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 11:06:25,336::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109438::WARNING::2014-10-04 11:07:25,335::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 11:07:25,336::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 11:07:25,336::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 11:07:35,337::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 11:07:35,338::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 11:07:35,338::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-04 11:08:27,612::__init__::232::IOProcess::(_processLogs) (92180) Queuing response >Thread-18::DEBUG::2014-10-04 11:08:27,613::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 11:08:27,613::__init__::232::IOProcess::(_processLogs) (92240) Got request for method 'glob' >Thread-18::DEBUG::2014-10-04 11:08:27,613::__init__::232::IOProcess::(_processLogs) (92181) Queuing response >Thread-18::DEBUG::2014-10-04 11:08:27,613::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 11:08:27,613::__init__::232::IOProcess::(_processLogs) (92241) Got request for method 'glob' >Thread-109439::WARNING::2014-10-04 11:08:35,337::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 11:08:35,338::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 11:08:35,338::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 11:08:45,339::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 11:08:45,340::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109440::WARNING::2014-10-04 11:09:45,339::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 11:09:45,340::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 11:09:45,340::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 11:09:55,341::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 11:09:55,341::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109441::WARNING::2014-10-04 11:10:55,341::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 11:10:55,342::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 11:10:55,342::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 11:11:05,343::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 11:11:05,343::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109442::WARNING::2014-10-04 11:12:05,343::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 11:12:05,343::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 11:12:05,344::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 11:12:15,345::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 11:12:15,345::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109443::WARNING::2014-10-04 11:13:15,345::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 11:13:15,345::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 11:13:15,346::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 11:13:25,346::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 11:13:25,347::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 11:13:25,347::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109444::WARNING::2014-10-04 11:14:25,347::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 11:14:25,347::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 11:14:25,348::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 11:14:35,349::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 11:14:35,349::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109445::WARNING::2014-10-04 11:15:35,349::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 11:15:35,349::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 11:15:35,349::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 11:15:45,351::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 11:15:45,351::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109446::WARNING::2014-10-04 11:16:45,351::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 11:16:45,351::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 11:16:45,351::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 11:16:55,352::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 11:16:55,353::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109447::WARNING::2014-10-04 11:17:55,352::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 11:17:55,353::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 11:17:55,353::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 11:18:05,354::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 11:18:05,354::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-04 11:18:31,772::__init__::232::IOProcess::(_processLogs) (92183) Queuing response >Thread-18::DEBUG::2014-10-04 11:18:31,773::__init__::232::IOProcess::(_processLogs) (92182) Queuing response >Thread-18::DEBUG::2014-10-04 11:18:31,773::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 11:18:31,773::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 11:18:31,773::__init__::232::IOProcess::(_processLogs) (92243) Got request for method 'glob' >Thread-18::DEBUG::2014-10-04 11:18:31,773::__init__::232::IOProcess::(_processLogs) (92242) Got request for method 'glob' >Thread-109448::WARNING::2014-10-04 11:19:05,354::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 11:19:05,355::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 11:19:05,355::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 11:19:15,355::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 11:19:15,356::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 11:19:15,356::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109449::WARNING::2014-10-04 11:20:15,356::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 11:20:15,357::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 11:20:15,357::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 11:20:25,358::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 11:20:25,358::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109450::WARNING::2014-10-04 11:21:25,358::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 11:21:25,358::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 11:21:25,359::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 11:21:35,360::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 11:21:35,360::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109451::WARNING::2014-10-04 11:22:35,360::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 11:22:35,360::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 11:22:35,361::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 11:22:45,362::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 11:22:45,362::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109452::WARNING::2014-10-04 11:23:45,362::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 11:23:45,362::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 11:23:45,362::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 11:23:55,364::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 11:23:55,364::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109453::WARNING::2014-10-04 11:24:55,364::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 11:24:55,364::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 11:24:55,364::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 11:25:05,365::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 11:25:05,366::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 11:25:05,366::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109454::WARNING::2014-10-04 11:26:05,366::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 11:26:05,366::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 11:26:05,366::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 11:26:15,367::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 11:26:15,368::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109455::WARNING::2014-10-04 11:27:15,367::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 11:27:15,368::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 11:27:15,368::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 11:27:25,369::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 11:27:25,369::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109456::WARNING::2014-10-04 11:28:25,369::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 11:28:25,370::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 11:28:25,370::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 11:28:35,371::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 11:28:35,371::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-04 11:28:35,932::__init__::232::IOProcess::(_processLogs) (92184) Queuing response >Thread-18::DEBUG::2014-10-04 11:28:35,932::__init__::232::IOProcess::(_processLogs) (92185) Queuing response >Thread-18::DEBUG::2014-10-04 11:28:35,933::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 11:28:35,933::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 11:28:35,933::__init__::232::IOProcess::(_processLogs) (92244) Got request for method 'glob' >Thread-18::DEBUG::2014-10-04 11:28:35,933::__init__::232::IOProcess::(_processLogs) (92245) Got request for method 'glob' >Thread-109457::WARNING::2014-10-04 11:29:35,371::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 11:29:35,371::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 11:29:35,371::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 11:29:45,373::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 11:29:45,373::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109458::WARNING::2014-10-04 11:30:45,373::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 11:30:45,373::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 11:30:45,373::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 11:30:55,374::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 11:30:55,375::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 11:30:55,375::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109459::WARNING::2014-10-04 11:31:55,374::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 11:31:55,375::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 11:31:55,375::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 11:32:05,376::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 11:32:05,376::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109460::WARNING::2014-10-04 11:33:05,376::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 11:33:05,377::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 11:33:05,377::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 11:33:15,378::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 11:33:15,378::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109461::WARNING::2014-10-04 11:34:15,378::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 11:34:15,378::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 11:34:15,379::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 11:34:25,380::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 11:34:25,380::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109462::WARNING::2014-10-04 11:35:25,380::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 11:35:25,380::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 11:35:25,380::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 11:35:35,382::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 11:35:35,382::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109463::WARNING::2014-10-04 11:36:35,381::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 11:36:35,382::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 11:36:35,382::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 11:36:45,382::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 11:36:45,383::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 11:36:45,384::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109464::WARNING::2014-10-04 11:37:45,383::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 11:37:45,384::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 11:37:45,384::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 11:37:55,385::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 11:37:55,385::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-04 11:38:42,140::__init__::232::IOProcess::(_processLogs) (92186) Queuing response >Thread-18::DEBUG::2014-10-04 11:38:42,141::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 11:38:42,141::__init__::232::IOProcess::(_processLogs) (92246) Got request for method 'glob' >Thread-109465::WARNING::2014-10-04 11:38:55,385::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 11:38:55,386::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 11:38:55,386::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 11:39:05,387::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 11:39:05,387::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109466::WARNING::2014-10-04 11:40:05,387::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 11:40:05,387::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 11:40:05,387::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 11:40:15,389::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 11:40:15,389::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109467::WARNING::2014-10-04 11:41:15,389::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 11:41:15,389::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 11:41:15,389::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 11:41:25,390::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 11:41:25,391::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109468::WARNING::2014-10-04 11:42:25,390::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 11:42:25,391::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 11:42:25,391::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 11:42:35,391::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 11:42:35,392::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 11:42:35,392::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109469::WARNING::2014-10-04 11:43:35,392::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 11:43:35,393::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 11:43:35,393::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 11:43:45,394::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 11:43:45,394::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109470::WARNING::2014-10-04 11:44:45,394::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 11:44:45,395::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 11:44:45,395::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 11:44:55,396::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 11:44:55,396::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109471::WARNING::2014-10-04 11:45:55,396::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 11:45:55,397::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 11:45:55,397::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 11:46:05,398::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 11:46:05,398::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109472::WARNING::2014-10-04 11:47:05,398::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 11:47:05,398::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 11:47:05,398::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 11:47:15,400::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 11:47:15,400::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109473::WARNING::2014-10-04 11:48:15,400::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 11:48:15,400::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 11:48:15,400::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 11:48:25,401::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 11:48:25,401::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 11:48:25,402::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-04 11:48:46,300::__init__::232::IOProcess::(_processLogs) (92187) Queuing response >Thread-18::DEBUG::2014-10-04 11:48:46,301::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 11:48:46,301::__init__::232::IOProcess::(_processLogs) (92247) Got request for method 'glob' >Thread-109474::WARNING::2014-10-04 11:49:25,401::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 11:49:25,402::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 11:49:25,402::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 11:49:35,403::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 11:49:35,403::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109475::WARNING::2014-10-04 11:50:35,403::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 11:50:35,404::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 11:50:35,404::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 11:50:45,405::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 11:50:45,406::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109476::WARNING::2014-10-04 11:51:45,405::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 11:51:45,406::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 11:51:45,406::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 11:51:55,407::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 11:51:55,408::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109477::WARNING::2014-10-04 11:52:55,407::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 11:52:55,408::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 11:52:55,408::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 11:53:05,409::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 11:53:05,409::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109478::WARNING::2014-10-04 11:54:05,409::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 11:54:05,410::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 11:54:05,410::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 11:54:15,410::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 11:54:15,411::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 11:54:15,412::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109479::WARNING::2014-10-04 11:55:15,411::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 11:55:15,412::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 11:55:15,412::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 11:55:25,413::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 11:55:25,413::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109480::WARNING::2014-10-04 11:56:25,413::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 11:56:25,414::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 11:56:25,414::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 11:56:35,415::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 11:56:35,415::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109481::WARNING::2014-10-04 11:57:35,415::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 11:57:35,415::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 11:57:35,416::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 11:57:45,417::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 11:57:45,417::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109482::WARNING::2014-10-04 11:58:45,417::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 11:58:45,417::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 11:58:45,418::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 11:58:50,460::__init__::232::IOProcess::(_processLogs) (92188) Queuing response >Thread-18::DEBUG::2014-10-04 11:58:50,461::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 11:58:50,462::__init__::232::IOProcess::(_processLogs) (92248) Got request for method 'glob' >Thread-18::DEBUG::2014-10-04 11:58:50,462::__init__::232::IOProcess::(_processLogs) (92190) Queuing response >Thread-18::DEBUG::2014-10-04 11:58:50,463::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 11:58:50,463::__init__::232::IOProcess::(_processLogs) (92249) Got request for method 'glob' >Thread-18::DEBUG::2014-10-04 11:58:55,419::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 11:58:55,419::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109483::WARNING::2014-10-04 11:59:55,419::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 11:59:55,419::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 11:59:55,419::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 12:00:05,420::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 12:00:05,421::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 12:00:05,421::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109484::WARNING::2014-10-04 12:01:05,421::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 12:01:05,421::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 12:01:05,421::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 12:01:15,423::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 12:01:15,423::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109485::WARNING::2014-10-04 12:02:15,423::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 12:02:15,423::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 12:02:15,423::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 12:02:25,424::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 12:02:25,425::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109486::WARNING::2014-10-04 12:03:25,424::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 12:03:25,425::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 12:03:25,425::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 12:03:35,426::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 12:03:35,427::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109487::WARNING::2014-10-04 12:04:35,426::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 12:04:35,427::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 12:04:35,427::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 12:04:45,428::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 12:04:45,428::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109488::WARNING::2014-10-04 12:05:45,428::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 12:05:45,429::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 12:05:45,429::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 12:05:55,429::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 12:05:55,430::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 12:05:55,430::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109489::WARNING::2014-10-04 12:06:55,430::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 12:06:55,431::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 12:06:55,431::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 12:07:05,432::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 12:07:05,432::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109490::WARNING::2014-10-04 12:08:05,432::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 12:08:05,432::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 12:08:05,433::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 12:08:15,434::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 12:08:15,434::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-04 12:08:54,620::__init__::232::IOProcess::(_processLogs) (92189) Queuing response >Thread-18::DEBUG::2014-10-04 12:08:54,621::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 12:08:54,621::__init__::232::IOProcess::(_processLogs) (92250) Got request for method 'glob' >Thread-18::DEBUG::2014-10-04 12:08:54,621::__init__::232::IOProcess::(_processLogs) (92191) Queuing response >Thread-18::DEBUG::2014-10-04 12:08:54,621::__init__::232::IOProcess::(_processLogs) (92192) Queuing response >Thread-18::DEBUG::2014-10-04 12:08:54,621::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 12:08:54,621::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 12:08:54,621::__init__::232::IOProcess::(_processLogs) (92251) Got request for method 'glob' >Thread-18::DEBUG::2014-10-04 12:08:54,621::__init__::232::IOProcess::(_processLogs) (92252) Got request for method 'glob' >Thread-109491::WARNING::2014-10-04 12:09:15,434::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 12:09:15,434::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 12:09:15,435::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 12:09:25,436::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 12:09:25,436::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109492::WARNING::2014-10-04 12:10:25,436::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 12:10:25,436::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 12:10:25,436::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 12:10:35,438::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 12:10:35,438::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109493::WARNING::2014-10-04 12:11:35,437::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 12:11:35,438::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 12:11:35,438::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 12:11:45,439::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 12:11:45,440::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 12:11:45,440::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109494::WARNING::2014-10-04 12:12:45,439::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 12:12:45,440::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 12:12:45,440::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 12:12:55,442::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 12:12:55,442::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109495::WARNING::2014-10-04 12:13:55,441::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 12:13:55,442::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 12:13:55,442::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 12:14:05,443::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 12:14:05,444::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109496::WARNING::2014-10-04 12:15:05,443::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 12:15:05,444::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 12:15:05,444::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 12:15:15,445::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 12:15:15,445::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109497::WARNING::2014-10-04 12:16:15,445::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 12:16:15,446::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 12:16:15,446::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 12:16:25,447::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 12:16:25,447::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109498::WARNING::2014-10-04 12:17:25,447::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 12:17:25,448::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 12:17:25,448::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 12:17:35,448::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 12:17:35,449::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 12:17:35,450::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109499::WARNING::2014-10-04 12:18:35,449::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 12:18:35,450::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 12:18:35,450::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 12:18:45,451::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 12:18:45,452::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-04 12:18:58,780::__init__::232::IOProcess::(_processLogs) (92193) Queuing response >Thread-18::DEBUG::2014-10-04 12:18:58,781::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 12:18:58,781::__init__::232::IOProcess::(_processLogs) (92253) Got request for method 'glob' >Thread-18::DEBUG::2014-10-04 12:18:58,781::__init__::232::IOProcess::(_processLogs) (92194) Queuing response >Thread-18::DEBUG::2014-10-04 12:18:58,781::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 12:18:58,781::__init__::232::IOProcess::(_processLogs) (92254) Got request for method 'glob' >Thread-109500::WARNING::2014-10-04 12:19:45,451::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 12:19:45,452::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 12:19:45,452::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 12:19:55,453::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 12:19:55,454::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109501::WARNING::2014-10-04 12:20:55,453::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 12:20:55,454::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 12:20:55,454::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 12:21:05,455::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 12:21:05,456::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109502::WARNING::2014-10-04 12:22:05,455::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 12:22:05,456::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 12:22:05,456::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 12:22:15,457::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 12:22:15,458::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109503::WARNING::2014-10-04 12:23:15,457::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 12:23:15,458::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 12:23:15,458::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 12:23:25,458::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 12:23:25,459::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 12:23:25,459::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109504::WARNING::2014-10-04 12:24:25,459::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 12:24:25,460::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 12:24:25,460::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 12:24:35,461::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 12:24:35,461::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109505::WARNING::2014-10-04 12:25:35,461::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 12:25:35,461::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 12:25:35,462::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 12:25:45,463::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 12:25:45,463::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109506::WARNING::2014-10-04 12:26:45,463::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 12:26:45,463::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 12:26:45,463::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 12:26:55,465::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 12:26:55,465::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109507::WARNING::2014-10-04 12:27:55,464::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 12:27:55,465::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 12:27:55,465::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 12:28:05,466::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 12:28:05,466::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-04 12:29:02,940::__init__::232::IOProcess::(_processLogs) (92195) Queuing response >Thread-18::DEBUG::2014-10-04 12:29:02,941::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 12:29:02,941::__init__::232::IOProcess::(_processLogs) (92255) Got request for method 'glob' >Thread-18::DEBUG::2014-10-04 12:29:02,941::__init__::232::IOProcess::(_processLogs) (92196) Queuing response >Thread-18::DEBUG::2014-10-04 12:29:02,941::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 12:29:02,941::__init__::232::IOProcess::(_processLogs) (92256) Got request for method 'glob' >Thread-109508::WARNING::2014-10-04 12:29:05,466::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 12:29:05,466::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 12:29:05,467::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 12:29:15,467::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 12:29:15,468::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 12:29:15,468::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109509::WARNING::2014-10-04 12:30:15,468::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 12:30:15,468::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 12:30:15,469::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 12:30:25,470::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 12:30:25,470::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109510::WARNING::2014-10-04 12:31:25,470::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 12:31:25,470::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 12:31:25,470::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 12:31:35,471::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 12:31:35,472::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109511::WARNING::2014-10-04 12:32:35,471::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 12:32:35,472::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 12:32:35,472::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 12:32:45,473::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 12:32:45,473::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109512::WARNING::2014-10-04 12:33:45,473::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 12:33:45,474::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 12:33:45,474::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 12:33:55,475::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 12:33:55,475::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109513::WARNING::2014-10-04 12:34:55,475::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 12:34:55,475::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 12:34:55,475::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 12:35:05,476::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 12:35:05,477::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 12:35:05,477::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109514::WARNING::2014-10-04 12:36:05,477::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 12:36:05,477::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 12:36:05,477::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 12:36:15,478::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 12:36:15,479::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109515::WARNING::2014-10-04 12:37:15,478::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 12:37:15,479::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 12:37:15,479::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 12:37:25,480::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 12:37:25,480::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109516::WARNING::2014-10-04 12:38:25,480::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 12:38:25,481::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 12:38:25,481::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 12:38:35,482::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 12:38:35,482::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-04 12:39:07,100::__init__::232::IOProcess::(_processLogs) (92198) Queuing response >Thread-18::DEBUG::2014-10-04 12:39:07,101::__init__::232::IOProcess::(_processLogs) (92197) Queuing response >Thread-18::DEBUG::2014-10-04 12:39:07,101::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 12:39:07,101::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 12:39:07,101::__init__::232::IOProcess::(_processLogs) (92257) Got request for method 'glob' >Thread-18::DEBUG::2014-10-04 12:39:07,101::__init__::232::IOProcess::(_processLogs) (92258) Got request for method 'glob' >Thread-109517::WARNING::2014-10-04 12:39:35,482::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 12:39:35,482::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 12:39:35,482::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 12:39:45,484::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 12:39:45,484::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109518::WARNING::2014-10-04 12:40:45,484::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 12:40:45,484::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 12:40:45,484::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 12:40:55,485::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 12:40:55,486::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 12:40:55,486::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109519::WARNING::2014-10-04 12:41:55,486::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 12:41:55,486::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 12:41:55,486::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 12:42:05,487::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 12:42:05,488::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109520::WARNING::2014-10-04 12:43:05,487::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 12:43:05,488::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 12:43:05,488::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 12:43:15,489::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 12:43:15,490::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109521::WARNING::2014-10-04 12:44:15,490::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 12:44:15,491::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 12:44:15,492::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 12:44:25,494::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 12:44:25,494::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109522::WARNING::2014-10-04 12:45:25,494::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 12:45:25,494::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 12:45:25,494::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 12:45:35,496::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 12:45:35,496::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109523::WARNING::2014-10-04 12:46:35,496::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 12:46:35,496::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 12:46:35,496::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 12:46:45,497::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 12:46:45,498::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 12:46:45,498::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109524::WARNING::2014-10-04 12:47:45,498::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 12:47:45,498::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 12:47:45,499::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 12:47:55,500::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 12:47:55,500::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109525::WARNING::2014-10-04 12:48:55,500::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 12:48:55,500::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 12:48:55,501::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 12:49:05,502::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 12:49:05,502::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-04 12:49:11,260::__init__::232::IOProcess::(_processLogs) (92199) Queuing response >Thread-18::DEBUG::2014-10-04 12:49:11,261::__init__::232::IOProcess::(_processLogs) (92201) Queuing response >Thread-18::DEBUG::2014-10-04 12:49:11,261::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 12:49:11,261::__init__::232::IOProcess::(_processLogs) (92259) Got request for method 'glob' >Thread-18::DEBUG::2014-10-04 12:49:11,261::__init__::232::IOProcess::(_processLogs) (92200) Queuing response >Thread-18::DEBUG::2014-10-04 12:49:11,261::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 12:49:11,261::__init__::232::IOProcess::(_processLogs) (92260) Got request for method 'glob' >Thread-18::DEBUG::2014-10-04 12:49:11,261::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 12:49:11,262::__init__::232::IOProcess::(_processLogs) (92261) Got request for method 'glob' >Thread-109526::WARNING::2014-10-04 12:50:05,502::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 12:50:05,502::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 12:50:05,503::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 12:50:15,504::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 12:50:15,504::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109527::WARNING::2014-10-04 12:51:15,504::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 12:51:15,504::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 12:51:15,505::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 12:51:25,506::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 12:51:25,506::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109528::WARNING::2014-10-04 12:52:25,506::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 12:52:25,506::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 12:52:25,506::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 12:52:35,507::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 12:52:35,509::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 12:52:35,509::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109529::WARNING::2014-10-04 12:53:35,509::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 12:53:35,509::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 12:53:35,509::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 12:53:45,511::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 12:53:45,511::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109530::WARNING::2014-10-04 12:54:45,510::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 12:54:45,511::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 12:54:45,511::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 12:54:55,512::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 12:54:55,513::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109531::WARNING::2014-10-04 12:55:55,512::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 12:55:55,513::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 12:55:55,513::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 12:56:05,514::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 12:56:05,514::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109532::WARNING::2014-10-04 12:57:05,514::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 12:57:05,515::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 12:57:05,515::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 12:57:15,516::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 12:57:15,516::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109533::WARNING::2014-10-04 12:58:15,516::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 12:58:15,517::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 12:58:15,517::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 12:58:25,517::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 12:58:25,518::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 12:58:25,518::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-04 12:59:15,420::__init__::232::IOProcess::(_processLogs) (92202) Queuing response >Thread-18::DEBUG::2014-10-04 12:59:15,421::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 12:59:15,421::__init__::232::IOProcess::(_processLogs) (92262) Got request for method 'glob' >Thread-18::DEBUG::2014-10-04 12:59:15,421::__init__::232::IOProcess::(_processLogs) (92203) Queuing response >Thread-18::DEBUG::2014-10-04 12:59:15,421::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 12:59:15,421::__init__::232::IOProcess::(_processLogs) (92263) Got request for method 'glob' >Thread-109534::WARNING::2014-10-04 12:59:25,518::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 12:59:25,519::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 12:59:25,519::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 12:59:35,520::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 12:59:35,521::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109535::WARNING::2014-10-04 13:00:35,520::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 13:00:35,521::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 13:00:35,521::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 13:00:45,522::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 13:00:45,522::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109536::WARNING::2014-10-04 13:01:45,522::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 13:01:45,523::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 13:01:45,523::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 13:01:55,524::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 13:01:55,525::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109537::WARNING::2014-10-04 13:02:55,524::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 13:02:55,525::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 13:02:55,525::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 13:03:05,526::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 13:03:05,526::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109538::WARNING::2014-10-04 13:04:05,526::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 13:04:05,527::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 13:04:05,527::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 13:04:15,527::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 13:04:15,528::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 13:04:15,528::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109539::WARNING::2014-10-04 13:05:15,528::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 13:05:15,529::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 13:05:15,529::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 13:05:25,530::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 13:05:25,530::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109540::WARNING::2014-10-04 13:06:25,530::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 13:06:25,531::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 13:06:25,531::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 13:06:35,532::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 13:06:35,532::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109541::WARNING::2014-10-04 13:07:35,532::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 13:07:35,533::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 13:07:35,533::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 13:07:45,534::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 13:07:45,534::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109542::WARNING::2014-10-04 13:08:45,534::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 13:08:45,535::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 13:08:45,535::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 13:08:55,536::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 13:08:55,536::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109543::WARNING::2014-10-04 13:09:55,536::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 13:09:55,537::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 13:09:55,537::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 13:10:05,537::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 13:10:05,538::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 13:10:05,538::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109544::WARNING::2014-10-04 13:11:05,538::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 13:11:05,539::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 13:11:05,539::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 13:11:15,540::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 13:11:15,540::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109545::WARNING::2014-10-04 13:12:15,540::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 13:12:15,541::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 13:12:15,541::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 13:12:25,542::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 13:12:25,542::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109546::WARNING::2014-10-04 13:13:25,542::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 13:13:25,542::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 13:13:25,543::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 13:13:35,544::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 13:13:35,544::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109547::WARNING::2014-10-04 13:14:35,544::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 13:14:35,544::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 13:14:35,545::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 13:14:45,546::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 13:14:45,546::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109548::WARNING::2014-10-04 13:15:45,546::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 13:15:45,546::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 13:15:45,547::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 13:15:55,547::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 13:15:55,548::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 13:15:55,548::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109549::WARNING::2014-10-04 13:16:55,548::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 13:16:55,549::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 13:16:55,549::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 13:17:05,550::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 13:17:05,550::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109550::WARNING::2014-10-04 13:18:05,550::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 13:18:05,551::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 13:18:05,551::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 13:18:15,552::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 13:18:15,552::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109551::WARNING::2014-10-04 13:19:15,552::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 13:19:15,553::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 13:19:15,553::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 13:19:25,554::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 13:19:25,554::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109552::WARNING::2014-10-04 13:20:25,554::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 13:20:25,555::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 13:20:25,555::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 13:20:35,556::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 13:20:35,556::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109553::WARNING::2014-10-04 13:21:35,556::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 13:21:35,557::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 13:21:35,557::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 13:21:45,557::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 13:21:45,558::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 13:21:45,559::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109554::WARNING::2014-10-04 13:22:45,558::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 13:22:45,559::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 13:22:45,559::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 13:22:55,561::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 13:22:55,561::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109555::WARNING::2014-10-04 13:23:55,560::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 13:23:55,561::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 13:23:55,561::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 13:24:05,562::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 13:24:05,562::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109556::WARNING::2014-10-04 13:25:05,562::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 13:25:05,563::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 13:25:05,563::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 13:25:15,564::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 13:25:15,564::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109557::WARNING::2014-10-04 13:26:15,564::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 13:26:15,564::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 13:26:15,565::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 13:26:25,566::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 13:26:25,566::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109558::WARNING::2014-10-04 13:27:25,566::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 13:27:25,566::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 13:27:25,566::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 13:27:35,567::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 13:27:35,568::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 13:27:35,568::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109559::WARNING::2014-10-04 13:28:35,568::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 13:28:35,568::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 13:28:35,568::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 13:28:45,569::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 13:28:45,570::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109560::WARNING::2014-10-04 13:29:45,569::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 13:29:45,570::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 13:29:45,570::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 13:29:55,571::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 13:29:55,571::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109561::WARNING::2014-10-04 13:30:55,571::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 13:30:55,571::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 13:30:55,572::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 13:31:05,573::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 13:31:05,573::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109562::WARNING::2014-10-04 13:32:05,573::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 13:32:05,573::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 13:32:05,573::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 13:32:15,574::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 13:32:15,575::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109563::WARNING::2014-10-04 13:33:15,574::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 13:33:15,575::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 13:33:15,575::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 13:33:25,575::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 13:33:25,576::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 13:33:25,577::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109564::WARNING::2014-10-04 13:34:25,576::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 13:34:25,577::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 13:34:25,577::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 13:34:35,578::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 13:34:35,578::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109565::WARNING::2014-10-04 13:35:35,578::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 13:35:35,579::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 13:35:35,579::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 13:35:45,580::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 13:35:45,580::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109566::WARNING::2014-10-04 13:36:45,580::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 13:36:45,580::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 13:36:45,580::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 13:36:55,582::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 13:36:55,582::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109567::WARNING::2014-10-04 13:37:55,582::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 13:37:55,582::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 13:37:55,582::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 13:38:05,583::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 13:38:05,584::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109568::WARNING::2014-10-04 13:39:05,583::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 13:39:05,584::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 13:39:05,584::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 13:39:15,584::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 13:39:15,585::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 13:39:15,585::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109569::WARNING::2014-10-04 13:40:15,585::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 13:40:15,586::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 13:40:15,586::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 13:40:25,587::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 13:40:25,587::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109570::WARNING::2014-10-04 13:41:25,587::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 13:41:25,588::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 13:41:25,588::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 13:41:35,589::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 13:41:35,589::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109571::WARNING::2014-10-04 13:42:35,589::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 13:42:35,589::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 13:42:35,590::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 13:42:45,591::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 13:42:45,591::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109572::WARNING::2014-10-04 13:43:45,591::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 13:43:45,591::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 13:43:45,591::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 13:43:55,593::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 13:43:55,593::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109573::WARNING::2014-10-04 13:44:55,593::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 13:44:55,593::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 13:44:55,593::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 13:45:05,594::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 13:45:05,595::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 13:45:05,595::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109574::WARNING::2014-10-04 13:46:05,595::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 13:46:05,595::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 13:46:05,596::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 13:46:15,597::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 13:46:15,597::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109575::WARNING::2014-10-04 13:47:15,597::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 13:47:15,597::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 13:47:15,598::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 13:47:25,599::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 13:47:25,599::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109576::WARNING::2014-10-04 13:48:25,599::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 13:48:25,599::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 13:48:25,600::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 13:48:35,601::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 13:48:35,601::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109577::WARNING::2014-10-04 13:49:35,601::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 13:49:35,601::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 13:49:35,602::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 13:49:45,603::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 13:49:45,603::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109578::WARNING::2014-10-04 13:50:45,603::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 13:50:45,603::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 13:50:45,603::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 13:50:55,604::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 13:50:55,605::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 13:50:55,605::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109579::WARNING::2014-10-04 13:51:55,605::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 13:51:55,606::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 13:51:55,606::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 13:52:05,607::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 13:52:05,607::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109580::WARNING::2014-10-04 13:53:05,607::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 13:53:05,608::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 13:53:05,608::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 13:53:15,612::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 13:53:15,613::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109581::WARNING::2014-10-04 13:54:15,612::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 13:54:15,613::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 13:54:15,613::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 13:54:25,614::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 13:54:25,614::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109582::WARNING::2014-10-04 13:55:25,614::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 13:55:25,614::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 13:55:25,615::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 13:55:35,616::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 13:55:35,616::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109583::WARNING::2014-10-04 13:56:35,616::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 13:56:35,616::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 13:56:35,616::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 13:56:45,617::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 13:56:45,618::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 13:56:45,618::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109584::WARNING::2014-10-04 13:57:45,618::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 13:57:45,618::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 13:57:45,619::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 13:57:55,620::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 13:57:55,620::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109585::WARNING::2014-10-04 13:58:55,620::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 13:58:55,620::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 13:58:55,620::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 13:59:05,622::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 13:59:05,622::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109586::WARNING::2014-10-04 14:00:05,621::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 14:00:05,622::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 14:00:05,622::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 14:00:15,623::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 14:00:15,624::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109587::WARNING::2014-10-04 14:01:15,623::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 14:01:15,624::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 14:01:15,624::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 14:01:25,625::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 14:01:25,625::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109588::WARNING::2014-10-04 14:02:25,625::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 14:02:25,626::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 14:02:25,626::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 14:02:35,626::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 14:02:35,627::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 14:02:35,627::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109589::WARNING::2014-10-04 14:03:35,627::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 14:03:35,628::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 14:03:35,628::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 14:03:45,629::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 14:03:45,629::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109590::WARNING::2014-10-04 14:04:45,629::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 14:04:45,629::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 14:04:45,630::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 14:04:55,631::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 14:04:55,631::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109591::WARNING::2014-10-04 14:05:55,631::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 14:05:55,631::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 14:05:55,631::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 14:06:05,633::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 14:06:05,633::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109592::WARNING::2014-10-04 14:07:05,633::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 14:07:05,633::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 14:07:05,633::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 14:07:15,634::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 14:07:15,635::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109593::WARNING::2014-10-04 14:08:15,634::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 14:08:15,635::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 14:08:15,635::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 14:08:25,635::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 14:08:25,636::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 14:08:25,637::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109594::WARNING::2014-10-04 14:09:25,636::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 14:09:25,637::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 14:09:25,637::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 14:09:35,638::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 14:09:35,639::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-04 14:09:44,540::__init__::232::IOProcess::(_processLogs) (92204) Queuing response >Thread-18::DEBUG::2014-10-04 14:09:44,541::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 14:09:44,541::__init__::232::IOProcess::(_processLogs) (92264) Got request for method 'glob' >Thread-109595::WARNING::2014-10-04 14:10:35,638::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 14:10:35,639::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 14:10:35,639::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 14:10:45,640::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 14:10:45,640::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109596::WARNING::2014-10-04 14:11:45,640::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 14:11:45,641::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 14:11:45,641::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 14:11:55,642::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 14:11:55,642::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109597::WARNING::2014-10-04 14:12:55,642::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 14:12:55,643::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 14:12:55,643::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 14:13:05,644::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 14:13:05,644::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109598::WARNING::2014-10-04 14:14:05,644::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 14:14:05,644::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 14:14:05,645::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 14:14:15,645::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 14:14:15,646::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 14:14:15,646::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109599::WARNING::2014-10-04 14:15:15,646::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 14:15:15,646::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 14:15:15,647::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 14:15:25,648::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 14:15:25,648::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109600::WARNING::2014-10-04 14:16:25,648::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 14:16:25,648::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 14:16:25,649::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 14:16:35,650::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 14:16:35,650::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109601::WARNING::2014-10-04 14:17:35,650::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 14:17:35,650::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 14:17:35,651::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 14:17:45,652::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 14:17:45,652::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109602::WARNING::2014-10-04 14:18:45,652::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 14:18:45,652::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 14:18:45,653::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 14:18:55,654::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 14:18:55,654::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109603::WARNING::2014-10-04 14:19:55,654::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 14:19:55,654::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 14:19:55,655::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 14:20:05,655::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 14:20:05,656::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 14:20:05,656::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109604::WARNING::2014-10-04 14:21:05,656::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 14:21:05,657::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 14:21:05,657::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 14:21:15,658::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 14:21:15,658::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109605::WARNING::2014-10-04 14:22:15,658::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 14:22:15,658::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 14:22:15,659::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 14:22:25,660::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 14:22:25,660::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109606::WARNING::2014-10-04 14:23:25,660::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 14:23:25,660::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 14:23:25,660::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 14:23:35,662::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 14:23:35,662::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109607::WARNING::2014-10-04 14:24:35,662::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 14:24:35,662::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 14:24:35,662::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 14:24:45,663::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 14:24:45,663::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109608::WARNING::2014-10-04 14:25:45,663::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 14:25:45,664::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 14:25:45,664::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 14:25:55,664::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 14:25:55,665::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 14:25:55,665::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109609::WARNING::2014-10-04 14:26:55,665::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 14:26:55,666::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 14:26:55,666::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 14:27:05,667::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 14:27:05,667::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109610::WARNING::2014-10-04 14:28:05,667::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 14:28:05,667::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 14:28:05,668::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 14:28:15,669::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 14:28:15,669::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109611::WARNING::2014-10-04 14:29:15,669::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 14:29:15,669::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 14:29:15,669::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 14:29:25,671::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 14:29:25,671::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109612::WARNING::2014-10-04 14:30:25,670::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 14:30:25,671::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 14:30:25,671::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 14:30:35,672::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 14:30:35,673::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109613::WARNING::2014-10-04 14:31:35,672::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 14:31:35,673::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 14:31:35,673::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 14:31:45,673::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 14:31:45,674::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 14:31:45,674::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109614::WARNING::2014-10-04 14:32:45,674::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 14:32:45,675::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 14:32:45,675::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 14:32:55,676::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 14:32:55,676::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109615::WARNING::2014-10-04 14:33:55,676::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 14:33:55,676::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 14:33:55,677::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 14:34:05,678::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 14:34:05,678::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109616::WARNING::2014-10-04 14:35:05,678::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 14:35:05,678::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 14:35:05,678::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 14:35:15,680::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 14:35:15,680::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109617::WARNING::2014-10-04 14:36:15,680::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 14:36:15,680::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 14:36:15,680::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 14:36:25,682::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 14:36:25,682::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109618::WARNING::2014-10-04 14:37:25,681::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 14:37:25,682::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 14:37:25,682::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 14:37:35,683::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 14:37:35,684::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 14:37:35,684::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109619::WARNING::2014-10-04 14:38:35,683::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 14:38:35,684::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 14:38:35,684::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 14:38:45,685::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 14:38:45,686::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109620::WARNING::2014-10-04 14:39:45,685::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 14:39:45,686::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 14:39:45,686::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 14:39:55,687::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 14:39:55,688::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109621::WARNING::2014-10-04 14:40:55,687::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 14:40:55,688::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 14:40:55,688::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 14:41:05,689::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 14:41:05,690::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109622::WARNING::2014-10-04 14:42:05,689::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 14:42:05,690::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 14:42:05,690::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 14:42:15,691::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 14:42:15,691::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109623::WARNING::2014-10-04 14:43:15,691::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 14:43:15,692::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 14:43:15,692::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 14:43:25,692::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 14:43:25,693::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 14:43:25,694::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109624::WARNING::2014-10-04 14:44:25,693::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 14:44:25,694::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 14:44:25,694::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 14:44:35,695::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 14:44:35,695::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109625::WARNING::2014-10-04 14:45:35,695::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 14:45:35,696::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 14:45:35,696::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 14:45:45,697::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 14:45:45,697::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109626::WARNING::2014-10-04 14:46:45,697::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 14:46:45,698::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 14:46:45,698::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 14:46:55,699::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 14:46:55,699::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109627::WARNING::2014-10-04 14:47:55,699::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 14:47:55,699::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 14:47:55,700::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 14:48:05,701::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 14:48:05,701::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109628::WARNING::2014-10-04 14:49:05,701::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 14:49:05,702::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 14:49:05,702::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 14:49:15,702::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 14:49:15,703::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 14:49:15,703::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109629::WARNING::2014-10-04 14:50:15,703::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 14:50:15,704::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 14:50:15,704::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 14:50:25,705::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 14:50:25,705::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109630::WARNING::2014-10-04 14:51:25,705::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 14:51:25,706::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 14:51:25,706::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 14:51:35,707::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 14:51:35,707::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109631::WARNING::2014-10-04 14:52:35,707::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 14:52:35,708::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 14:52:35,708::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 14:52:45,709::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 14:52:45,709::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109632::WARNING::2014-10-04 14:53:45,709::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 14:53:45,709::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 14:53:45,710::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 14:53:55,711::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 14:53:55,711::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109633::WARNING::2014-10-04 14:54:55,711::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 14:54:55,711::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 14:54:55,712::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 14:55:05,712::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 14:55:05,713::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 14:55:05,713::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109634::WARNING::2014-10-04 14:56:05,713::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 14:56:05,713::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 14:56:05,714::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 14:56:15,715::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 14:56:15,715::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109635::WARNING::2014-10-04 14:57:15,715::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 14:57:15,715::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 14:57:15,715::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 14:57:25,717::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 14:57:25,717::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109636::WARNING::2014-10-04 14:58:25,716::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 14:58:25,717::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 14:58:25,717::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 14:58:35,718::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 14:58:35,719::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109637::WARNING::2014-10-04 14:59:35,718::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 14:59:35,719::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 14:59:35,719::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 14:59:45,720::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 14:59:45,720::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-04 15:00:05,340::__init__::232::IOProcess::(_processLogs) (92205) Queuing response >Thread-18::DEBUG::2014-10-04 15:00:05,341::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 15:00:05,341::__init__::232::IOProcess::(_processLogs) (92265) Got request for method 'glob' >Thread-109638::WARNING::2014-10-04 15:00:45,720::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 15:00:45,721::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 15:00:45,721::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 15:00:55,721::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 15:00:55,722::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 15:00:55,722::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109639::WARNING::2014-10-04 15:01:55,722::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 15:01:55,723::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 15:01:55,723::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 15:02:05,724::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 15:02:05,724::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109640::WARNING::2014-10-04 15:03:05,724::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 15:03:05,725::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 15:03:05,725::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 15:03:15,726::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 15:03:15,726::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109641::WARNING::2014-10-04 15:04:15,726::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 15:04:15,726::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 15:04:15,727::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 15:04:25,728::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 15:04:25,728::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109642::WARNING::2014-10-04 15:05:25,728::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 15:05:25,728::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 15:05:25,729::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 15:05:35,730::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 15:05:35,730::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109643::WARNING::2014-10-04 15:06:35,730::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 15:06:35,730::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 15:06:35,730::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 15:06:45,731::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 15:06:45,732::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 15:06:45,732::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109644::WARNING::2014-10-04 15:07:45,731::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 15:07:45,732::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 15:07:45,732::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 15:07:55,734::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 15:07:55,734::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109645::WARNING::2014-10-04 15:08:55,733::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 15:08:55,734::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 15:08:55,734::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 15:09:05,735::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 15:09:05,736::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109646::WARNING::2014-10-04 15:10:05,735::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 15:10:05,736::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 15:10:05,736::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 15:10:09,500::__init__::232::IOProcess::(_processLogs) (92206) Queuing response >Thread-18::DEBUG::2014-10-04 15:10:09,501::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 15:10:09,501::__init__::232::IOProcess::(_processLogs) (92266) Got request for method 'glob' >Thread-18::DEBUG::2014-10-04 15:10:15,737::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 15:10:15,737::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109647::WARNING::2014-10-04 15:11:15,737::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 15:11:15,738::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 15:11:15,738::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 15:11:25,739::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 15:11:25,739::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109648::WARNING::2014-10-04 15:12:25,739::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 15:12:25,740::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 15:12:25,740::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 15:12:35,740::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 15:12:35,741::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 15:12:35,741::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109649::WARNING::2014-10-04 15:13:35,741::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 15:13:35,742::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 15:13:35,742::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 15:13:45,743::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 15:13:45,743::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109650::WARNING::2014-10-04 15:14:45,743::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 15:14:45,744::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 15:14:45,744::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 15:14:55,745::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 15:14:55,745::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109651::WARNING::2014-10-04 15:15:55,745::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 15:15:55,745::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 15:15:55,746::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 15:16:05,747::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 15:16:05,747::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109652::WARNING::2014-10-04 15:17:05,747::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 15:17:05,747::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 15:17:05,748::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 15:17:15,749::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 15:17:15,749::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109653::WARNING::2014-10-04 15:18:15,749::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 15:18:15,749::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 15:18:15,749::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 15:18:25,750::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 15:18:25,751::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 15:18:25,751::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109654::WARNING::2014-10-04 15:19:25,751::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 15:19:25,751::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 15:19:25,752::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 15:19:35,753::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 15:19:35,753::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-04 15:20:13,660::__init__::232::IOProcess::(_processLogs) (92207) Queuing response >Thread-18::DEBUG::2014-10-04 15:20:13,661::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 15:20:13,661::__init__::232::IOProcess::(_processLogs) (92267) Got request for method 'glob' >Thread-109655::WARNING::2014-10-04 15:20:35,753::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 15:20:35,753::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 15:20:35,754::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 15:20:45,755::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 15:20:45,755::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109656::WARNING::2014-10-04 15:21:45,755::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 15:21:45,755::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 15:21:45,755::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 15:21:55,757::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 15:21:55,757::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109657::WARNING::2014-10-04 15:22:55,757::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 15:22:55,757::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 15:22:55,757::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 15:23:05,759::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 15:23:05,759::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109658::WARNING::2014-10-04 15:24:05,759::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 15:24:05,759::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 15:24:05,759::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 15:24:15,760::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 15:24:15,760::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 15:24:15,761::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109659::WARNING::2014-10-04 15:25:15,760::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 15:25:15,761::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 15:25:15,761::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 15:25:25,762::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 15:25:25,762::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109660::WARNING::2014-10-04 15:26:25,762::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 15:26:25,763::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 15:26:25,763::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 15:26:35,764::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 15:26:35,764::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109661::WARNING::2014-10-04 15:27:35,764::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 15:27:35,764::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 15:27:35,765::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 15:27:45,766::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 15:27:45,766::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109662::WARNING::2014-10-04 15:28:45,766::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 15:28:45,766::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 15:28:45,766::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 15:28:55,767::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 15:28:55,768::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109663::WARNING::2014-10-04 15:29:55,767::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 15:29:55,768::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 15:29:55,768::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 15:30:05,769::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 15:30:05,769::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 15:30:05,770::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109664::WARNING::2014-10-04 15:31:05,769::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 15:31:05,770::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 15:31:05,770::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 15:31:15,771::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 15:31:15,772::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109665::WARNING::2014-10-04 15:32:15,771::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 15:32:15,772::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 15:32:15,772::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 15:32:25,773::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 15:32:25,773::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109666::WARNING::2014-10-04 15:33:25,773::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 15:33:25,774::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 15:33:25,774::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 15:33:35,775::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 15:33:35,775::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109667::WARNING::2014-10-04 15:34:35,775::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 15:34:35,776::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 15:34:35,776::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 15:34:45,777::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 15:34:45,777::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109668::WARNING::2014-10-04 15:35:45,777::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 15:35:45,778::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 15:35:45,778::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 15:35:55,778::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 15:35:55,779::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 15:35:55,779::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109669::WARNING::2014-10-04 15:36:55,779::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 15:36:55,780::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 15:36:55,780::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 15:37:05,781::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 15:37:05,781::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109670::WARNING::2014-10-04 15:38:05,781::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 15:38:05,782::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 15:38:05,782::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 15:38:15,783::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 15:38:15,783::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109671::WARNING::2014-10-04 15:39:15,783::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 15:39:15,783::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 15:39:15,784::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 15:39:25,785::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 15:39:25,785::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109672::WARNING::2014-10-04 15:40:25,785::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 15:40:25,785::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 15:40:25,785::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 15:40:35,787::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 15:40:35,787::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109673::WARNING::2014-10-04 15:41:35,786::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 15:41:35,787::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 15:41:35,787::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 15:41:45,788::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 15:41:45,789::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 15:41:45,789::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109674::WARNING::2014-10-04 15:42:45,788::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 15:42:45,789::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 15:42:45,789::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 15:42:55,790::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 15:42:55,791::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109675::WARNING::2014-10-04 15:43:55,790::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 15:43:55,791::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 15:43:55,791::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 15:44:05,792::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 15:44:05,792::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109676::WARNING::2014-10-04 15:45:05,792::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 15:45:05,793::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 15:45:05,793::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 15:45:15,794::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 15:45:15,794::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109677::WARNING::2014-10-04 15:46:15,794::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 15:46:15,795::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 15:46:15,795::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 15:46:25,796::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 15:46:25,796::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109678::WARNING::2014-10-04 15:47:25,796::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 15:47:25,796::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 15:47:25,797::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 15:47:35,797::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 15:47:35,798::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 15:47:35,798::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109679::WARNING::2014-10-04 15:48:35,798::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 15:48:35,799::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 15:48:35,799::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 15:48:45,800::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 15:48:45,800::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109680::WARNING::2014-10-04 15:49:45,800::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 15:49:45,801::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 15:49:45,801::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 15:49:55,802::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 15:49:55,802::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109681::WARNING::2014-10-04 15:50:55,802::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 15:50:55,803::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 15:50:55,803::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 15:51:05,804::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 15:51:05,804::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109682::WARNING::2014-10-04 15:52:05,804::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 15:52:05,805::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 15:52:05,805::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 15:52:15,806::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 15:52:15,806::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109683::WARNING::2014-10-04 15:53:15,806::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 15:53:15,807::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 15:53:15,807::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 15:53:25,807::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 15:53:25,808::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 15:53:25,808::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109684::WARNING::2014-10-04 15:54:25,808::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 15:54:25,809::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 15:54:25,809::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 15:54:35,810::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 15:54:35,810::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109685::WARNING::2014-10-04 15:55:35,810::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 15:55:35,811::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 15:55:35,811::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 15:55:45,812::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 15:55:45,812::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109686::WARNING::2014-10-04 15:56:45,812::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 15:56:45,812::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 15:56:45,813::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 15:56:55,814::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 15:56:55,814::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109687::WARNING::2014-10-04 15:57:55,814::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 15:57:55,814::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 15:57:55,814::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 15:58:05,816::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 15:58:05,816::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109688::WARNING::2014-10-04 15:59:05,816::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 15:59:05,816::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 15:59:05,816::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 15:59:15,817::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 15:59:15,818::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 15:59:15,818::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109689::WARNING::2014-10-04 16:00:15,818::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 16:00:15,818::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 16:00:15,818::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 16:00:25,819::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 16:00:25,820::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109690::WARNING::2014-10-04 16:01:25,819::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 16:01:25,820::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 16:01:25,820::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 16:01:35,821::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 16:01:35,822::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109691::WARNING::2014-10-04 16:02:35,821::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 16:02:35,822::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 16:02:35,822::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 16:02:45,823::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 16:02:45,823::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109692::WARNING::2014-10-04 16:03:45,823::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 16:03:45,824::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 16:03:45,824::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 16:03:55,825::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 16:03:55,825::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109693::WARNING::2014-10-04 16:04:55,825::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 16:04:55,826::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 16:04:55,826::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 16:05:05,826::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 16:05:05,827::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 16:05:05,827::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109694::WARNING::2014-10-04 16:06:05,827::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 16:06:05,828::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 16:06:05,828::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 16:06:15,829::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 16:06:15,829::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109695::WARNING::2014-10-04 16:07:15,829::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 16:07:15,829::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 16:07:15,830::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 16:07:25,831::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 16:07:25,831::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109696::WARNING::2014-10-04 16:08:25,831::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 16:08:25,831::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 16:08:25,831::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 16:08:35,833::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 16:08:35,833::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109697::WARNING::2014-10-04 16:09:35,833::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 16:09:35,833::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 16:09:35,833::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 16:09:45,835::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 16:09:45,835::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-04 16:10:34,460::__init__::232::IOProcess::(_processLogs) (92208) Queuing response >Thread-18::DEBUG::2014-10-04 16:10:34,461::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 16:10:34,461::__init__::232::IOProcess::(_processLogs) (92268) Got request for method 'glob' >Thread-109698::WARNING::2014-10-04 16:10:45,834::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 16:10:45,835::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 16:10:45,835::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 16:10:55,836::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 16:10:55,837::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 16:10:55,837::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109699::WARNING::2014-10-04 16:11:55,836::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 16:11:55,837::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 16:11:55,837::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 16:12:05,838::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 16:12:05,839::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109700::WARNING::2014-10-04 16:13:05,838::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 16:13:05,839::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 16:13:05,839::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 16:13:15,840::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 16:13:15,840::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109701::WARNING::2014-10-04 16:14:15,840::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 16:14:15,841::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 16:14:15,841::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 16:14:25,842::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 16:14:25,842::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109702::WARNING::2014-10-04 16:15:25,842::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 16:15:25,843::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 16:15:25,843::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 16:15:35,844::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 16:15:35,844::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109703::WARNING::2014-10-04 16:16:35,844::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 16:16:35,845::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 16:16:35,845::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 16:16:45,845::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 16:16:45,846::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 16:16:45,846::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109704::WARNING::2014-10-04 16:17:45,846::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 16:17:45,847::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 16:17:45,847::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 16:17:55,848::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 16:17:55,848::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109705::WARNING::2014-10-04 16:18:55,848::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 16:18:55,849::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 16:18:55,849::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 16:19:05,850::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 16:19:05,850::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109706::WARNING::2014-10-04 16:20:05,850::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 16:20:05,851::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 16:20:05,851::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 16:20:15,852::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 16:20:15,852::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-04 16:20:38,620::__init__::232::IOProcess::(_processLogs) (92209) Queuing response >Thread-18::DEBUG::2014-10-04 16:20:38,621::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 16:20:38,621::__init__::232::IOProcess::(_processLogs) (92269) Got request for method 'glob' >Thread-109707::WARNING::2014-10-04 16:21:15,852::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 16:21:15,852::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 16:21:15,853::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 16:21:25,854::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 16:21:25,854::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109708::WARNING::2014-10-04 16:22:25,854::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 16:22:25,854::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 16:22:25,855::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 16:22:35,855::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 16:22:35,856::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 16:22:35,856::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109709::WARNING::2014-10-04 16:23:35,856::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 16:23:35,857::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 16:23:35,857::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 16:23:45,858::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 16:23:45,858::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109710::WARNING::2014-10-04 16:24:45,858::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 16:24:45,859::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 16:24:45,859::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 16:24:55,860::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 16:24:55,860::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109711::WARNING::2014-10-04 16:25:55,860::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 16:25:55,860::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 16:25:55,861::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 16:26:05,862::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 16:26:05,862::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109712::WARNING::2014-10-04 16:27:05,862::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 16:27:05,862::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 16:27:05,863::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 16:27:15,864::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 16:27:15,864::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109713::WARNING::2014-10-04 16:28:15,864::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 16:28:15,864::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 16:28:15,864::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 16:28:25,865::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 16:28:25,866::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 16:28:25,866::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109714::WARNING::2014-10-04 16:29:25,866::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 16:29:25,866::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 16:29:25,866::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 16:29:35,868::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 16:29:35,868::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109715::WARNING::2014-10-04 16:30:35,868::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 16:30:35,868::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 16:30:35,868::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 16:30:45,869::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 16:30:45,870::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109716::WARNING::2014-10-04 16:31:45,869::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 16:31:45,870::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 16:31:45,870::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 16:31:55,871::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 16:31:55,872::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109717::WARNING::2014-10-04 16:32:55,871::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 16:32:55,872::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 16:32:55,872::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 16:33:05,873::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 16:33:05,874::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109718::WARNING::2014-10-04 16:34:05,873::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 16:34:05,874::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 16:34:05,874::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 16:34:15,874::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 16:34:15,875::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 16:34:15,876::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109719::WARNING::2014-10-04 16:35:15,875::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 16:35:15,876::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 16:35:15,876::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 16:35:25,877::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 16:35:25,877::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109720::WARNING::2014-10-04 16:36:25,877::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 16:36:25,878::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 16:36:25,878::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 16:36:35,879::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 16:36:35,879::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109721::WARNING::2014-10-04 16:37:35,879::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 16:37:35,880::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 16:37:35,880::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 16:37:45,881::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 16:37:45,881::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109722::WARNING::2014-10-04 16:38:45,881::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 16:38:45,881::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 16:38:45,882::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 16:38:55,883::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 16:38:55,883::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109723::WARNING::2014-10-04 16:39:55,883::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 16:39:55,883::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 16:39:55,883::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 16:40:05,884::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 16:40:05,885::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 16:40:05,885::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109724::WARNING::2014-10-04 16:41:05,885::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 16:41:05,885::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 16:41:05,885::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 16:41:15,886::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 16:41:15,887::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109725::WARNING::2014-10-04 16:42:15,886::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 16:42:15,887::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 16:42:15,887::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 16:42:25,888::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 16:42:25,888::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109726::WARNING::2014-10-04 16:43:25,888::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 16:43:25,889::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 16:43:25,889::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 16:43:35,890::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 16:43:35,890::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109727::WARNING::2014-10-04 16:44:35,890::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 16:44:35,891::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 16:44:35,891::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 16:44:45,892::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 16:44:45,892::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109728::WARNING::2014-10-04 16:45:45,892::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 16:45:45,892::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 16:45:45,893::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 16:45:55,893::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 16:45:55,894::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 16:45:55,894::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109729::WARNING::2014-10-04 16:46:55,894::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 16:46:55,894::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 16:46:55,895::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 16:47:05,896::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 16:47:05,896::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109730::WARNING::2014-10-04 16:48:05,896::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 16:48:05,896::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 16:48:05,896::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 16:48:15,898::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 16:48:15,898::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109731::WARNING::2014-10-04 16:49:15,898::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 16:49:15,898::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 16:49:15,898::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 16:49:25,900::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 16:49:25,900::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109732::WARNING::2014-10-04 16:50:25,899::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 16:50:25,900::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 16:50:25,900::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 16:50:35,901::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 16:50:35,902::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109733::WARNING::2014-10-04 16:51:35,901::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 16:51:35,902::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 16:51:35,902::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 16:51:45,902::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 16:51:45,903::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 16:51:45,904::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109734::WARNING::2014-10-04 16:52:45,903::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 16:52:45,904::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 16:52:45,904::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 16:52:55,905::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 16:52:55,906::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109735::WARNING::2014-10-04 16:53:55,905::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 16:53:55,906::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 16:53:55,906::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 16:54:05,907::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 16:54:05,907::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109736::WARNING::2014-10-04 16:55:05,907::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 16:55:05,907::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 16:55:05,908::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 16:55:15,909::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 16:55:15,909::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109737::WARNING::2014-10-04 16:56:15,909::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 16:56:15,909::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 16:56:15,910::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 16:56:25,911::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 16:56:25,911::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109738::WARNING::2014-10-04 16:57:25,911::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 16:57:25,911::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 16:57:25,911::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 16:57:35,912::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 16:57:35,913::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 16:57:35,913::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109739::WARNING::2014-10-04 16:58:35,912::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 16:58:35,913::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 16:58:35,913::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 16:58:45,914::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 16:58:45,914::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109740::WARNING::2014-10-04 16:59:45,914::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 16:59:45,915::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 16:59:45,915::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 16:59:55,916::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 16:59:55,916::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109741::WARNING::2014-10-04 17:00:55,916::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 17:00:55,916::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 17:00:55,917::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 17:01:05,918::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 17:01:05,918::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109742::WARNING::2014-10-04 17:02:05,918::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 17:02:05,918::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 17:02:05,918::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 17:02:15,920::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 17:02:15,920::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109743::WARNING::2014-10-04 17:03:15,919::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 17:03:15,920::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 17:03:15,920::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 17:03:25,921::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 17:03:25,921::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 17:03:25,922::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109744::WARNING::2014-10-04 17:04:25,921::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 17:04:25,922::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 17:04:25,922::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 17:04:35,923::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 17:04:35,923::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109745::WARNING::2014-10-04 17:05:35,923::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 17:05:35,924::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 17:05:35,924::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 17:05:45,925::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 17:05:45,925::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109746::WARNING::2014-10-04 17:06:45,925::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 17:06:45,926::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 17:06:45,926::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 17:06:55,927::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 17:06:55,927::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109747::WARNING::2014-10-04 17:07:55,927::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 17:07:55,927::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 17:07:55,927::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 17:08:05,929::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 17:08:05,929::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109748::WARNING::2014-10-04 17:09:05,929::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 17:09:05,929::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 17:09:05,929::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 17:09:15,930::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 17:09:15,931::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 17:09:15,931::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109749::WARNING::2014-10-04 17:10:15,930::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 17:10:15,931::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 17:10:15,931::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 17:10:25,932::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 17:10:25,933::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109750::WARNING::2014-10-04 17:11:25,932::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 17:11:25,933::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 17:11:25,933::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 17:11:35,934::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 17:11:35,934::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109751::WARNING::2014-10-04 17:12:35,934::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 17:12:35,934::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 17:12:35,935::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 17:12:45,936::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 17:12:45,936::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109752::WARNING::2014-10-04 17:13:45,936::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 17:13:45,936::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 17:13:45,936::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 17:13:55,937::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 17:13:55,938::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109753::WARNING::2014-10-04 17:14:55,937::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 17:14:55,938::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 17:14:55,938::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 17:15:05,938::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 17:15:05,939::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 17:15:05,939::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109754::WARNING::2014-10-04 17:16:05,939::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 17:16:05,940::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 17:16:05,940::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 17:16:15,941::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 17:16:15,941::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109755::WARNING::2014-10-04 17:17:15,941::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 17:17:15,941::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 17:17:15,941::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 17:17:25,942::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 17:17:25,943::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109756::WARNING::2014-10-04 17:18:25,942::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 17:18:25,943::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 17:18:25,943::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 17:18:35,944::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 17:18:35,944::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109757::WARNING::2014-10-04 17:19:35,944::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 17:19:35,945::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 17:19:35,945::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 17:19:45,946::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 17:19:45,946::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109758::WARNING::2014-10-04 17:20:45,946::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 17:20:45,947::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 17:20:45,947::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 17:20:55,947::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 17:20:55,948::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 17:20:55,948::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109759::WARNING::2014-10-04 17:21:55,948::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 17:21:55,949::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 17:21:55,949::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 17:22:05,950::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 17:22:05,950::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109760::WARNING::2014-10-04 17:23:05,950::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 17:23:05,951::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 17:23:05,951::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 17:23:15,952::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 17:23:15,952::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109761::WARNING::2014-10-04 17:24:15,952::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 17:24:15,952::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 17:24:15,953::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 17:24:25,954::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 17:24:25,954::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109762::WARNING::2014-10-04 17:25:25,954::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 17:25:25,954::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 17:25:25,954::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 17:25:35,955::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 17:25:35,956::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109763::WARNING::2014-10-04 17:26:35,955::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 17:26:35,956::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 17:26:35,956::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 17:26:45,957::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 17:26:45,958::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 17:26:45,958::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109764::WARNING::2014-10-04 17:27:45,957::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 17:27:45,958::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 17:27:45,958::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 17:27:55,959::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 17:27:55,959::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109765::WARNING::2014-10-04 17:28:55,959::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 17:28:55,960::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 17:28:55,960::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 17:29:05,961::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 17:29:05,961::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109766::WARNING::2014-10-04 17:30:05,961::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 17:30:05,962::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 17:30:05,962::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 17:30:15,963::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 17:30:15,963::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-04 17:31:07,741::__init__::232::IOProcess::(_processLogs) (92210) Queuing response >Thread-18::DEBUG::2014-10-04 17:31:07,741::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 17:31:07,741::__init__::232::IOProcess::(_processLogs) (92270) Got request for method 'glob' >Thread-109767::WARNING::2014-10-04 17:31:15,963::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 17:31:15,963::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 17:31:15,964::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 17:31:25,965::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 17:31:25,965::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109768::WARNING::2014-10-04 17:32:25,965::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 17:32:25,965::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 17:32:25,966::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 17:32:35,966::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 17:32:35,967::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 17:32:35,967::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109769::WARNING::2014-10-04 17:33:35,967::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 17:33:35,967::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 17:33:35,968::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 17:33:45,969::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 17:33:45,969::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109770::WARNING::2014-10-04 17:34:45,969::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 17:34:45,969::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 17:34:45,969::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 17:34:55,971::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 17:34:55,971::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109771::WARNING::2014-10-04 17:35:55,970::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 17:35:55,971::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 17:35:55,971::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 17:36:05,972::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 17:36:05,973::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109772::WARNING::2014-10-04 17:37:05,972::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 17:37:05,973::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 17:37:05,973::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 17:37:15,974::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 17:37:15,974::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109773::WARNING::2014-10-04 17:38:15,974::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 17:38:15,975::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 17:38:15,975::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 17:38:25,975::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 17:38:25,976::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 17:38:25,976::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109774::WARNING::2014-10-04 17:39:25,976::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 17:39:25,977::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 17:39:25,977::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 17:39:35,978::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 17:39:35,978::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109775::WARNING::2014-10-04 17:40:35,978::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 17:40:35,979::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 17:40:35,979::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 17:40:45,980::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 17:40:45,980::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-04 17:41:11,900::__init__::232::IOProcess::(_processLogs) (92211) Queuing response >Thread-18::DEBUG::2014-10-04 17:41:11,901::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 17:41:11,901::__init__::232::IOProcess::(_processLogs) (92271) Got request for method 'glob' >Thread-109776::WARNING::2014-10-04 17:41:45,980::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 17:41:45,980::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 17:41:45,981::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 17:41:55,982::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 17:41:55,982::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109777::WARNING::2014-10-04 17:42:55,982::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 17:42:55,982::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 17:42:55,982::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 17:43:05,984::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 17:43:05,984::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109778::WARNING::2014-10-04 17:44:05,984::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 17:44:05,984::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 17:44:05,984::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 17:44:15,985::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 17:44:15,986::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 17:44:15,986::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109779::WARNING::2014-10-04 17:45:15,986::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 17:45:15,986::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 17:45:15,986::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 17:45:25,988::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 17:45:25,988::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109780::WARNING::2014-10-04 17:46:25,987::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 17:46:25,988::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 17:46:25,988::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 17:46:35,989::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 17:46:35,989::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109781::WARNING::2014-10-04 17:47:35,989::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 17:47:35,990::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 17:47:35,990::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 17:47:45,991::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 17:47:45,992::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109782::WARNING::2014-10-04 17:48:45,991::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 17:48:45,992::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 17:48:45,992::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 17:48:55,993::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 17:48:55,993::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109783::WARNING::2014-10-04 17:49:55,993::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 17:49:55,994::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 17:49:55,994::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 17:50:05,994::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 17:50:05,995::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 17:50:05,995::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109784::WARNING::2014-10-04 17:51:05,995::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 17:51:05,996::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 17:51:05,996::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 17:51:15,997::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 17:51:15,997::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-04 17:51:16,060::__init__::232::IOProcess::(_processLogs) (92213) Queuing response >Thread-18::DEBUG::2014-10-04 17:51:16,060::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 17:51:16,060::__init__::232::IOProcess::(_processLogs) (92272) Got request for method 'glob' >Thread-18::DEBUG::2014-10-04 17:51:16,061::__init__::232::IOProcess::(_processLogs) (92212) Queuing response >Thread-18::DEBUG::2014-10-04 17:51:16,061::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 17:51:16,061::__init__::232::IOProcess::(_processLogs) (92273) Got request for method 'glob' >Thread-109785::WARNING::2014-10-04 17:52:15,997::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 17:52:15,998::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 17:52:15,998::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 17:52:25,999::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 17:52:25,999::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109786::WARNING::2014-10-04 17:53:25,999::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 17:53:26,000::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 17:53:26,000::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 17:53:36,001::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 17:53:36,001::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109787::WARNING::2014-10-04 17:54:36,001::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 17:54:36,002::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 17:54:36,002::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 17:54:46,003::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 17:54:46,004::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109788::WARNING::2014-10-04 17:55:46,003::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 17:55:46,003::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 17:55:46,003::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 17:55:56,004::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 17:55:56,005::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 17:55:56,005::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109789::WARNING::2014-10-04 17:56:56,005::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 17:56:56,005::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 17:56:56,005::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 17:57:06,007::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 17:57:06,007::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109790::WARNING::2014-10-04 17:58:06,007::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 17:58:06,007::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 17:58:06,007::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 17:58:16,008::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 17:58:16,008::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109791::WARNING::2014-10-04 17:59:16,008::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 17:59:16,009::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 17:59:16,009::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 17:59:26,011::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 17:59:26,011::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109792::WARNING::2014-10-04 18:00:26,010::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 18:00:26,011::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 18:00:26,011::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 18:00:36,012::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 18:00:36,012::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-04 18:01:22,268::__init__::232::IOProcess::(_processLogs) (92214) Queuing response >Thread-18::DEBUG::2014-10-04 18:01:22,269::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 18:01:22,269::__init__::232::IOProcess::(_processLogs) (92274) Got request for method 'glob' >Thread-18::DEBUG::2014-10-04 18:01:22,269::__init__::232::IOProcess::(_processLogs) (92215) Queuing response >Thread-18::DEBUG::2014-10-04 18:01:22,269::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 18:01:22,269::__init__::232::IOProcess::(_processLogs) (92275) Got request for method 'glob' >Thread-109793::WARNING::2014-10-04 18:01:36,012::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 18:01:36,013::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 18:01:36,013::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 18:01:46,013::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 18:01:46,014::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 18:01:46,014::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109794::WARNING::2014-10-04 18:02:46,014::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 18:02:46,015::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 18:02:46,015::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 18:02:56,016::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 18:02:56,016::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109795::WARNING::2014-10-04 18:03:56,016::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 18:03:56,017::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 18:03:56,017::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 18:04:06,018::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 18:04:06,018::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109796::WARNING::2014-10-04 18:05:06,018::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 18:05:06,018::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 18:05:06,018::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 18:05:16,020::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 18:05:16,020::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109797::WARNING::2014-10-04 18:06:16,019::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 18:06:16,020::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 18:06:16,020::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 18:06:26,021::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 18:06:26,022::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109798::WARNING::2014-10-04 18:07:26,021::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 18:07:26,022::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 18:07:26,022::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 18:07:36,022::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 18:07:36,023::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 18:07:36,023::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109799::WARNING::2014-10-04 18:08:36,023::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 18:08:36,024::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 18:08:36,024::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 18:08:46,025::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 18:08:46,025::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109800::WARNING::2014-10-04 18:09:46,025::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 18:09:46,025::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 18:09:46,025::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 18:09:56,027::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 18:09:56,027::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109801::WARNING::2014-10-04 18:10:56,027::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 18:10:56,027::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 18:10:56,027::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 18:11:06,028::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 18:11:06,029::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-04 18:11:26,428::__init__::232::IOProcess::(_processLogs) (92216) Queuing response >Thread-18::DEBUG::2014-10-04 18:11:26,429::__init__::232::IOProcess::(_processLogs) (92217) Queuing response >Thread-18::DEBUG::2014-10-04 18:11:26,429::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 18:11:26,429::__init__::232::IOProcess::(_processLogs) (92276) Got request for method 'glob' >Thread-18::DEBUG::2014-10-04 18:11:26,429::__init__::232::IOProcess::(_processLogs) (92218) Queuing response >Thread-18::DEBUG::2014-10-04 18:11:26,429::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 18:11:26,429::__init__::232::IOProcess::(_processLogs) (92277) Got request for method 'glob' >Thread-18::DEBUG::2014-10-04 18:11:26,429::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 18:11:26,429::__init__::232::IOProcess::(_processLogs) (92278) Got request for method 'glob' >Thread-109802::WARNING::2014-10-04 18:12:06,028::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 18:12:06,029::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 18:12:06,029::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 18:12:16,030::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 18:12:16,030::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109803::WARNING::2014-10-04 18:13:16,030::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 18:13:16,031::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 18:13:16,031::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 18:13:26,031::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 18:13:26,032::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 18:13:26,032::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109804::WARNING::2014-10-04 18:14:26,032::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 18:14:26,033::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 18:14:26,033::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 18:14:36,034::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 18:14:36,034::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109805::WARNING::2014-10-04 18:15:36,034::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 18:15:36,034::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 18:15:36,035::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 18:15:46,036::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 18:15:46,036::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109806::WARNING::2014-10-04 18:16:46,036::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 18:16:46,036::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 18:16:46,036::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 18:16:56,038::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 18:16:56,038::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109807::WARNING::2014-10-04 18:17:56,038::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 18:17:56,038::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 18:17:56,038::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 18:18:06,040::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 18:18:06,040::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109808::WARNING::2014-10-04 18:19:06,040::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 18:19:06,040::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 18:19:06,040::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 18:19:16,041::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 18:19:16,042::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 18:19:16,042::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109809::WARNING::2014-10-04 18:20:16,042::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 18:20:16,042::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 18:20:16,043::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 18:20:26,044::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 18:20:26,044::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109810::WARNING::2014-10-04 18:21:26,044::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 18:21:26,044::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 18:21:26,044::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 18:21:32,636::__init__::232::IOProcess::(_processLogs) (92219) Queuing response >Thread-18::DEBUG::2014-10-04 18:21:32,637::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 18:21:32,637::__init__::232::IOProcess::(_processLogs) (92279) Got request for method 'glob' >Thread-18::DEBUG::2014-10-04 18:21:32,637::__init__::232::IOProcess::(_processLogs) (92220) Queuing response >Thread-18::DEBUG::2014-10-04 18:21:32,637::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 18:21:32,637::__init__::232::IOProcess::(_processLogs) (92280) Got request for method 'glob' >Thread-18::DEBUG::2014-10-04 18:21:36,046::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 18:21:36,046::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109811::WARNING::2014-10-04 18:22:36,046::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 18:22:36,046::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 18:22:36,046::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 18:22:46,048::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 18:22:46,048::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109812::WARNING::2014-10-04 18:23:46,048::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 18:23:46,048::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 18:23:46,048::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 18:23:56,051::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 18:23:56,052::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109813::WARNING::2014-10-04 18:24:56,051::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 18:24:56,052::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 18:24:56,052::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 18:25:06,052::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 18:25:06,053::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 18:25:06,053::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109814::WARNING::2014-10-04 18:26:06,053::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 18:26:06,054::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 18:26:06,054::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 18:26:16,055::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 18:26:16,055::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109815::WARNING::2014-10-04 18:27:16,055::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 18:27:16,056::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 18:27:16,056::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 18:27:26,057::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 18:27:26,057::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109816::WARNING::2014-10-04 18:28:26,057::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 18:28:26,057::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 18:28:26,058::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 18:28:36,059::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 18:28:36,059::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109817::WARNING::2014-10-04 18:29:36,059::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 18:29:36,059::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 18:29:36,059::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 18:29:46,061::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 18:29:46,061::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109818::WARNING::2014-10-04 18:30:46,060::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 18:30:46,061::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 18:30:46,061::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 18:30:56,062::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 18:30:56,063::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 18:30:56,063::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-04 18:31:36,796::__init__::232::IOProcess::(_processLogs) (92221) Queuing response >Thread-18::DEBUG::2014-10-04 18:31:36,797::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 18:31:36,797::__init__::232::IOProcess::(_processLogs) (92281) Got request for method 'glob' >Thread-18::DEBUG::2014-10-04 18:31:36,797::__init__::232::IOProcess::(_processLogs) (92222) Queuing response >Thread-18::DEBUG::2014-10-04 18:31:36,797::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 18:31:36,797::__init__::232::IOProcess::(_processLogs) (92282) Got request for method 'glob' >Thread-109819::WARNING::2014-10-04 18:31:56,063::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 18:31:56,063::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 18:31:56,063::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 18:32:06,065::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 18:32:06,065::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109820::WARNING::2014-10-04 18:33:06,065::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 18:33:06,065::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 18:33:06,065::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 18:33:16,066::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 18:33:16,067::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109821::WARNING::2014-10-04 18:34:16,066::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 18:34:16,067::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 18:34:16,067::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 18:34:26,068::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 18:34:26,069::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109822::WARNING::2014-10-04 18:35:26,068::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 18:35:26,069::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 18:35:26,069::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 18:35:36,070::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 18:35:36,070::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109823::WARNING::2014-10-04 18:36:36,070::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 18:36:36,071::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 18:36:36,071::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 18:36:46,071::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 18:36:46,072::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 18:36:46,072::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109824::WARNING::2014-10-04 18:37:46,072::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 18:37:46,073::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 18:37:46,073::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 18:37:56,074::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 18:37:56,074::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109825::WARNING::2014-10-04 18:38:56,074::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 18:38:56,074::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 18:38:56,075::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 18:39:06,076::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 18:39:06,076::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109826::WARNING::2014-10-04 18:40:06,076::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 18:40:06,076::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 18:40:06,078::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 18:40:16,079::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 18:40:16,080::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109827::WARNING::2014-10-04 18:41:16,079::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 18:41:16,080::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 18:41:16,080::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 18:41:26,081::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 18:41:26,081::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-04 18:41:43,004::__init__::232::IOProcess::(_processLogs) (92223) Queuing response >Thread-18::DEBUG::2014-10-04 18:41:43,005::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 18:41:43,005::__init__::232::IOProcess::(_processLogs) (92283) Got request for method 'glob' >Thread-18::DEBUG::2014-10-04 18:41:43,005::__init__::232::IOProcess::(_processLogs) (92224) Queuing response >Thread-18::DEBUG::2014-10-04 18:41:43,005::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 18:41:43,005::__init__::232::IOProcess::(_processLogs) (92284) Got request for method 'glob' >Thread-109828::WARNING::2014-10-04 18:42:26,081::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 18:42:26,082::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 18:42:26,082::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 18:42:36,082::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 18:42:36,083::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 18:42:36,083::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109829::WARNING::2014-10-04 18:43:36,083::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 18:43:36,084::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 18:43:36,084::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 18:43:46,085::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 18:43:46,086::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109830::WARNING::2014-10-04 18:44:46,085::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 18:44:46,086::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 18:44:46,086::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 18:44:56,087::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 18:44:56,087::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109831::WARNING::2014-10-04 18:45:56,087::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 18:45:56,088::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 18:45:56,088::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 18:46:06,089::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 18:46:06,089::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109832::WARNING::2014-10-04 18:47:06,089::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 18:47:06,090::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 18:47:06,090::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 18:47:16,091::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 18:47:16,092::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109833::WARNING::2014-10-04 18:48:16,091::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 18:48:16,092::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 18:48:16,092::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 18:48:26,093::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 18:48:26,094::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 18:48:26,094::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109834::WARNING::2014-10-04 18:49:26,094::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 18:49:26,094::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 18:49:26,094::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 18:49:36,096::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 18:49:36,096::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109835::WARNING::2014-10-04 18:50:36,095::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 18:50:36,096::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 18:50:36,096::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 18:50:46,098::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 18:50:46,098::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109836::WARNING::2014-10-04 18:51:46,097::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 18:51:46,098::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 18:51:46,098::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 18:51:47,164::__init__::232::IOProcess::(_processLogs) (92225) Queuing response >Thread-18::DEBUG::2014-10-04 18:51:47,164::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 18:51:47,165::__init__::232::IOProcess::(_processLogs) (92226) Queuing response >Thread-18::DEBUG::2014-10-04 18:51:47,165::__init__::232::IOProcess::(_processLogs) (92285) Got request for method 'glob' >Thread-18::DEBUG::2014-10-04 18:51:47,165::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 18:51:47,165::__init__::232::IOProcess::(_processLogs) (92286) Got request for method 'glob' >Thread-18::DEBUG::2014-10-04 18:51:56,099::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 18:51:56,100::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109837::WARNING::2014-10-04 18:52:56,099::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 18:52:56,100::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 18:52:56,100::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 18:53:06,101::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 18:53:06,102::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109838::WARNING::2014-10-04 18:54:06,101::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 18:54:06,102::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 18:54:06,103::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 18:54:16,103::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 18:54:16,104::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 18:54:16,104::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109839::WARNING::2014-10-04 18:55:16,104::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 18:55:16,104::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 18:55:16,105::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 18:55:26,106::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 18:55:26,106::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109840::WARNING::2014-10-04 18:56:26,106::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 18:56:26,106::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 18:56:26,106::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 18:56:36,108::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 18:56:36,108::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109841::WARNING::2014-10-04 18:57:36,108::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 18:57:36,108::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 18:57:36,108::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 18:57:46,109::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 18:57:46,110::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109842::WARNING::2014-10-04 18:58:46,109::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 18:58:46,110::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 18:58:46,110::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 18:58:56,111::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 18:58:56,112::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109843::WARNING::2014-10-04 18:59:56,111::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 18:59:56,112::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 18:59:56,112::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 19:00:06,112::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 19:00:06,113::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 19:00:06,113::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109844::WARNING::2014-10-04 19:01:06,113::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 19:01:06,114::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 19:01:06,114::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 19:01:16,115::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 19:01:16,115::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-04 19:01:51,324::__init__::232::IOProcess::(_processLogs) (92227) Queuing response >Thread-18::DEBUG::2014-10-04 19:01:51,325::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 19:01:51,325::__init__::232::IOProcess::(_processLogs) (92287) Got request for method 'glob' >Thread-18::DEBUG::2014-10-04 19:01:51,325::__init__::232::IOProcess::(_processLogs) (92228) Queuing response >Thread-18::DEBUG::2014-10-04 19:01:51,325::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 19:01:51,325::__init__::232::IOProcess::(_processLogs) (92288) Got request for method 'glob' >Thread-109845::WARNING::2014-10-04 19:02:16,115::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 19:02:16,115::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 19:02:16,116::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 19:02:26,117::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 19:02:26,117::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109846::WARNING::2014-10-04 19:03:26,117::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 19:03:26,117::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 19:03:26,117::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 19:03:36,119::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 19:03:36,119::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109847::WARNING::2014-10-04 19:04:36,119::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 19:04:36,119::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 19:04:36,119::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 19:04:46,120::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 19:04:46,120::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109848::WARNING::2014-10-04 19:05:46,120::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 19:05:46,121::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 19:05:46,121::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 19:05:56,121::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 19:05:56,122::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 19:05:56,122::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109849::WARNING::2014-10-04 19:06:56,122::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 19:06:56,123::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 19:06:56,123::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 19:07:06,124::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 19:07:06,124::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109850::WARNING::2014-10-04 19:08:06,124::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 19:08:06,124::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 19:08:06,125::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 19:08:16,127::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 19:08:16,128::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109851::WARNING::2014-10-04 19:09:16,127::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 19:09:16,128::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 19:09:16,128::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 19:09:26,129::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 19:09:26,129::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109852::WARNING::2014-10-04 19:10:26,129::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 19:10:26,129::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 19:10:26,130::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 19:10:36,131::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 19:10:36,131::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109853::WARNING::2014-10-04 19:11:36,131::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 19:11:36,132::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 19:11:36,132::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 19:11:46,132::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 19:11:46,133::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 19:11:46,133::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-04 19:11:55,484::__init__::232::IOProcess::(_processLogs) (92229) Queuing response >Thread-18::DEBUG::2014-10-04 19:11:55,485::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 19:11:55,485::__init__::232::IOProcess::(_processLogs) (92289) Got request for method 'glob' >Thread-18::DEBUG::2014-10-04 19:11:55,485::__init__::232::IOProcess::(_processLogs) (92230) Queuing response >Thread-18::DEBUG::2014-10-04 19:11:55,485::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 19:11:55,485::__init__::232::IOProcess::(_processLogs) (92290) Got request for method 'glob' >Thread-109854::WARNING::2014-10-04 19:12:46,133::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 19:12:46,134::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 19:12:46,134::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 19:12:56,135::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 19:12:56,135::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109855::WARNING::2014-10-04 19:13:56,135::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 19:13:56,136::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 19:13:56,136::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 19:14:06,137::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 19:14:06,137::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109856::WARNING::2014-10-04 19:15:06,137::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 19:15:06,138::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 19:15:06,138::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 19:15:16,139::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 19:15:16,139::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109857::WARNING::2014-10-04 19:16:16,139::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 19:16:16,140::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 19:16:16,140::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 19:16:26,141::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 19:16:26,141::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109858::WARNING::2014-10-04 19:17:26,141::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 19:17:26,142::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 19:17:26,142::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 19:17:36,142::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 19:17:36,143::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 19:17:36,143::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109859::WARNING::2014-10-04 19:18:36,143::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 19:18:36,144::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 19:18:36,144::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 19:18:46,145::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 19:18:46,145::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109860::WARNING::2014-10-04 19:19:46,145::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 19:19:46,146::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 19:19:46,146::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 19:19:56,147::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 19:19:56,147::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109861::WARNING::2014-10-04 19:20:56,147::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 19:20:56,148::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 19:20:56,148::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 19:21:06,149::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 19:21:06,149::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-04 19:21:59,644::__init__::232::IOProcess::(_processLogs) (92231) Queuing response >Thread-18::DEBUG::2014-10-04 19:21:59,645::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 19:21:59,645::__init__::232::IOProcess::(_processLogs) (92291) Got request for method 'glob' >Thread-18::DEBUG::2014-10-04 19:21:59,645::__init__::232::IOProcess::(_processLogs) (92232) Queuing response >Thread-18::DEBUG::2014-10-04 19:21:59,645::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 19:21:59,645::__init__::232::IOProcess::(_processLogs) (92233) Queuing response >Thread-18::DEBUG::2014-10-04 19:21:59,645::__init__::232::IOProcess::(_processLogs) (92292) Got request for method 'glob' >Thread-18::DEBUG::2014-10-04 19:21:59,645::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 19:21:59,645::__init__::232::IOProcess::(_processLogs) (92293) Got request for method 'glob' >Thread-109862::WARNING::2014-10-04 19:22:06,149::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 19:22:06,150::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 19:22:06,150::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 19:22:16,151::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 19:22:16,151::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109863::WARNING::2014-10-04 19:23:16,151::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 19:23:16,152::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 19:23:16,152::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 19:23:26,152::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 19:23:26,153::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 19:23:26,153::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109864::WARNING::2014-10-04 19:24:26,153::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 19:24:26,154::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 19:24:26,154::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 19:24:36,155::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 19:24:36,155::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109865::WARNING::2014-10-04 19:25:36,155::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 19:25:36,156::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 19:25:36,156::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 19:25:46,157::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 19:25:46,157::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109866::WARNING::2014-10-04 19:26:46,157::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 19:26:46,157::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 19:26:46,158::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 19:26:56,159::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 19:26:56,159::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109867::WARNING::2014-10-04 19:27:56,159::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 19:27:56,159::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 19:27:56,160::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 19:28:06,161::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 19:28:06,161::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109868::WARNING::2014-10-04 19:29:06,161::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 19:29:06,161::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 19:29:06,161::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 19:29:16,162::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 19:29:16,163::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 19:29:16,163::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109869::WARNING::2014-10-04 19:30:16,163::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 19:30:16,163::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 19:30:16,163::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 19:30:26,165::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 19:30:26,165::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109870::WARNING::2014-10-04 19:31:26,165::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 19:31:26,165::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 19:31:26,165::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 19:31:36,167::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 19:31:36,167::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-04 19:32:03,804::__init__::232::IOProcess::(_processLogs) (92234) Queuing response >Thread-18::DEBUG::2014-10-04 19:32:03,805::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 19:32:03,805::__init__::232::IOProcess::(_processLogs) (92294) Got request for method 'glob' >Thread-18::DEBUG::2014-10-04 19:32:03,805::__init__::232::IOProcess::(_processLogs) (92235) Queuing response >Thread-18::DEBUG::2014-10-04 19:32:03,805::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 19:32:03,805::__init__::232::IOProcess::(_processLogs) (92295) Got request for method 'glob' >Thread-109871::WARNING::2014-10-04 19:32:36,167::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 19:32:36,167::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 19:32:36,167::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 19:32:46,168::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 19:32:46,169::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109872::WARNING::2014-10-04 19:33:46,168::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 19:33:46,169::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 19:33:46,169::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 19:33:56,170::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 19:33:56,170::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109873::WARNING::2014-10-04 19:34:56,170::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 19:34:56,171::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 19:34:56,171::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 19:35:06,171::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 19:35:06,172::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 19:35:06,172::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109874::WARNING::2014-10-04 19:36:06,172::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 19:36:06,173::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 19:36:06,173::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 19:36:16,174::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 19:36:16,174::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109875::WARNING::2014-10-04 19:37:16,174::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 19:37:16,175::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 19:37:16,175::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 19:37:26,176::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 19:37:26,176::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109876::WARNING::2014-10-04 19:38:26,176::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 19:38:26,176::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 19:38:26,177::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 19:38:36,178::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 19:38:36,178::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109877::WARNING::2014-10-04 19:39:36,178::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 19:39:36,178::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 19:39:36,179::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 19:39:46,180::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 19:39:46,180::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109878::WARNING::2014-10-04 19:40:46,180::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 19:40:46,180::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 19:40:46,181::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 19:40:56,181::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 19:40:56,182::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 19:40:56,182::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109879::WARNING::2014-10-04 19:41:56,182::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 19:41:56,183::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 19:41:56,183::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 19:42:06,184::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 19:42:06,184::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-04 19:42:07,964::__init__::232::IOProcess::(_processLogs) (92236) Queuing response >Thread-18::DEBUG::2014-10-04 19:42:07,964::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 19:42:07,965::__init__::232::IOProcess::(_processLogs) (92296) Got request for method 'glob' >Thread-109880::WARNING::2014-10-04 19:43:06,184::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 19:43:06,185::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 19:43:06,185::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 19:43:16,186::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 19:43:16,187::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109881::WARNING::2014-10-04 19:44:16,186::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 19:44:16,187::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 19:44:16,187::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 19:44:26,188::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 19:44:26,188::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109882::WARNING::2014-10-04 19:45:26,188::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 19:45:26,189::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 19:45:26,189::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 19:45:36,190::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 19:45:36,190::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109883::WARNING::2014-10-04 19:46:36,190::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 19:46:36,191::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 19:46:36,191::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 19:46:46,191::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 19:46:46,192::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 19:46:46,192::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109884::WARNING::2014-10-04 19:47:46,192::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 19:47:46,193::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 19:47:46,193::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 19:47:56,194::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 19:47:56,194::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109885::WARNING::2014-10-04 19:48:56,194::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 19:48:56,195::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 19:48:56,195::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 19:49:06,196::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 19:49:06,196::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109886::WARNING::2014-10-04 19:50:06,196::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 19:50:06,197::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 19:50:06,197::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 19:50:16,198::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 19:50:16,198::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109887::WARNING::2014-10-04 19:51:16,198::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 19:51:16,199::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 19:51:16,199::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 19:51:26,200::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 19:51:26,200::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-04 19:52:14,172::__init__::232::IOProcess::(_processLogs) (92237) Queuing response >Thread-18::DEBUG::2014-10-04 19:52:14,173::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 19:52:14,173::__init__::232::IOProcess::(_processLogs) (92297) Got request for method 'glob' >Thread-109888::WARNING::2014-10-04 19:52:26,200::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 19:52:26,201::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 19:52:26,201::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 19:52:36,201::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 19:52:36,202::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 19:52:36,202::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109889::WARNING::2014-10-04 19:53:36,202::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 19:53:36,203::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 19:53:36,203::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 19:53:46,204::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 19:53:46,205::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109890::WARNING::2014-10-04 19:54:46,204::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 19:54:46,205::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 19:54:46,205::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 19:54:56,206::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 19:54:56,206::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109891::WARNING::2014-10-04 19:55:56,206::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 19:55:56,207::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 19:55:56,207::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 19:56:06,208::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 19:56:06,208::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109892::WARNING::2014-10-04 19:57:06,208::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 19:57:06,208::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 19:57:06,209::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 19:57:16,210::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 19:57:16,210::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109893::WARNING::2014-10-04 19:58:16,210::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 19:58:16,210::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 19:58:16,210::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 19:58:26,211::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 19:58:26,212::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 19:58:26,212::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109894::WARNING::2014-10-04 19:59:26,212::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 19:59:26,212::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 19:59:26,212::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 19:59:36,214::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 19:59:36,214::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109895::WARNING::2014-10-04 20:00:36,213::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 20:00:36,214::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 20:00:36,214::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 20:00:46,215::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 20:00:46,216::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109896::WARNING::2014-10-04 20:01:46,215::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 20:01:46,216::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 20:01:46,216::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 20:01:56,217::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 20:01:56,217::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-04 20:02:18,333::__init__::232::IOProcess::(_processLogs) (92238) Queuing response >Thread-18::DEBUG::2014-10-04 20:02:18,333::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 20:02:18,333::__init__::232::IOProcess::(_processLogs) (92298) Got request for method 'glob' >Thread-18::DEBUG::2014-10-04 20:02:18,333::__init__::232::IOProcess::(_processLogs) (92239) Queuing response >Thread-18::DEBUG::2014-10-04 20:02:18,333::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 20:02:18,333::__init__::232::IOProcess::(_processLogs) (92299) Got request for method 'glob' >Thread-109897::WARNING::2014-10-04 20:02:56,217::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 20:02:56,218::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 20:02:56,218::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 20:03:06,219::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 20:03:06,219::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109898::WARNING::2014-10-04 20:04:06,219::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 20:04:06,219::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 20:04:06,220::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 20:04:16,220::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 20:04:16,221::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 20:04:16,221::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109899::WARNING::2014-10-04 20:05:16,221::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 20:05:16,222::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 20:05:16,222::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 20:05:26,223::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 20:05:26,223::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109900::WARNING::2014-10-04 20:06:26,223::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 20:06:26,224::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 20:06:26,224::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 20:06:36,225::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 20:06:36,225::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109901::WARNING::2014-10-04 20:07:36,225::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 20:07:36,225::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 20:07:36,226::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 20:07:46,227::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 20:07:46,227::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109902::WARNING::2014-10-04 20:08:46,227::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 20:08:46,227::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 20:08:46,227::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 20:08:56,229::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 20:08:56,229::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109903::WARNING::2014-10-04 20:09:56,229::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 20:09:56,229::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 20:09:56,229::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 20:10:06,230::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 20:10:06,231::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 20:10:06,231::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109904::WARNING::2014-10-04 20:11:06,231::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 20:11:06,231::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 20:11:06,231::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 20:11:16,233::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 20:11:16,233::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109905::WARNING::2014-10-04 20:12:16,232::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 20:12:16,233::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 20:12:16,233::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 20:12:22,492::__init__::232::IOProcess::(_processLogs) (92240) Queuing response >Thread-18::DEBUG::2014-10-04 20:12:22,493::__init__::232::IOProcess::(_processLogs) (92241) Queuing response >Thread-18::DEBUG::2014-10-04 20:12:22,493::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 20:12:22,493::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 20:12:22,493::__init__::232::IOProcess::(_processLogs) (92300) Got request for method 'glob' >Thread-18::DEBUG::2014-10-04 20:12:22,493::__init__::232::IOProcess::(_processLogs) (92301) Got request for method 'glob' >Thread-18::DEBUG::2014-10-04 20:12:26,234::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 20:12:26,235::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109906::WARNING::2014-10-04 20:13:26,234::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 20:13:26,235::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 20:13:26,235::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 20:13:36,236::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 20:13:36,236::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109907::WARNING::2014-10-04 20:14:36,236::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 20:14:36,237::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 20:14:36,237::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 20:14:46,238::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 20:14:46,239::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109908::WARNING::2014-10-04 20:15:46,238::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 20:15:46,239::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 20:15:46,239::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 20:15:56,240::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 20:15:56,240::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 20:15:56,241::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109909::WARNING::2014-10-04 20:16:56,240::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 20:16:56,241::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 20:16:56,241::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 20:17:06,242::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 20:17:06,243::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109910::WARNING::2014-10-04 20:18:06,242::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 20:18:06,243::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 20:18:06,243::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 20:18:16,244::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 20:18:16,245::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109911::WARNING::2014-10-04 20:19:16,244::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 20:19:16,245::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 20:19:16,245::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 20:19:26,246::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 20:19:26,247::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109912::WARNING::2014-10-04 20:20:26,246::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 20:20:26,247::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 20:20:26,247::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 20:20:36,248::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 20:20:36,249::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109913::WARNING::2014-10-04 20:21:36,248::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 20:21:36,249::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 20:21:36,249::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 20:21:46,250::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 20:21:46,251::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 20:21:46,251::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-04 20:22:28,700::__init__::232::IOProcess::(_processLogs) (92242) Queuing response >Thread-18::DEBUG::2014-10-04 20:22:28,701::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 20:22:28,701::__init__::232::IOProcess::(_processLogs) (92302) Got request for method 'glob' >Thread-18::DEBUG::2014-10-04 20:22:28,701::__init__::232::IOProcess::(_processLogs) (92243) Queuing response >Thread-18::DEBUG::2014-10-04 20:22:28,701::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 20:22:28,701::__init__::232::IOProcess::(_processLogs) (92303) Got request for method 'glob' >Thread-109914::WARNING::2014-10-04 20:22:46,251::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 20:22:46,251::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 20:22:46,252::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 20:22:56,253::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 20:22:56,253::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109915::WARNING::2014-10-04 20:23:56,253::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 20:23:56,253::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 20:23:56,253::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 20:24:06,255::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 20:24:06,255::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109916::WARNING::2014-10-04 20:25:06,255::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 20:25:06,255::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 20:25:06,255::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 20:25:16,257::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 20:25:16,257::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109917::WARNING::2014-10-04 20:26:16,256::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 20:26:16,257::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 20:26:16,257::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 20:26:26,258::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 20:26:26,259::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109918::WARNING::2014-10-04 20:27:26,258::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 20:27:26,259::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 20:27:26,259::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 20:27:36,259::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 20:27:36,260::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 20:27:36,261::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109919::WARNING::2014-10-04 20:28:36,260::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 20:28:36,261::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 20:28:36,261::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 20:28:46,263::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 20:28:46,263::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109920::WARNING::2014-10-04 20:29:46,263::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 20:29:46,263::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 20:29:46,263::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 20:29:56,265::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 20:29:56,265::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109921::WARNING::2014-10-04 20:30:56,265::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 20:30:56,265::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 20:30:56,265::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 20:31:06,267::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 20:31:06,267::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109922::WARNING::2014-10-04 20:32:06,266::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 20:32:06,267::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 20:32:06,267::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 20:32:16,268::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 20:32:16,269::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-04 20:32:32,860::__init__::232::IOProcess::(_processLogs) (92245) Queuing response >Thread-18::DEBUG::2014-10-04 20:32:32,861::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 20:32:32,861::__init__::232::IOProcess::(_processLogs) (92304) Got request for method 'glob' >Thread-18::DEBUG::2014-10-04 20:32:32,861::__init__::232::IOProcess::(_processLogs) (92244) Queuing response >Thread-18::DEBUG::2014-10-04 20:32:32,861::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 20:32:32,861::__init__::232::IOProcess::(_processLogs) (92305) Got request for method 'glob' >Thread-109923::WARNING::2014-10-04 20:33:16,268::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 20:33:16,269::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 20:33:16,269::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 20:33:26,269::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 20:33:26,270::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 20:33:26,271::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109924::WARNING::2014-10-04 20:34:26,270::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 20:34:26,271::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 20:34:26,271::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 20:34:36,272::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 20:34:36,272::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109925::WARNING::2014-10-04 20:35:36,272::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 20:35:36,273::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 20:35:36,273::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 20:35:46,274::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 20:35:46,274::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109926::WARNING::2014-10-04 20:36:46,274::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 20:36:46,275::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 20:36:46,275::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 20:36:56,276::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 20:36:56,276::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109927::WARNING::2014-10-04 20:37:56,276::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 20:37:56,277::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 20:37:56,277::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 20:38:06,278::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 20:38:06,278::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109928::WARNING::2014-10-04 20:39:06,278::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 20:39:06,278::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 20:39:06,279::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 20:39:16,279::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 20:39:16,280::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 20:39:16,280::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109929::WARNING::2014-10-04 20:40:16,280::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 20:40:16,280::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 20:40:16,281::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 20:40:26,282::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 20:40:26,282::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109930::WARNING::2014-10-04 20:41:26,282::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 20:41:26,282::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 20:41:26,283::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 20:41:36,284::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 20:41:36,284::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109931::WARNING::2014-10-04 20:42:36,284::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 20:42:36,284::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 20:42:36,284::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 20:42:37,020::__init__::232::IOProcess::(_processLogs) (92246) Queuing response >Thread-18::DEBUG::2014-10-04 20:42:37,021::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 20:42:37,021::__init__::232::IOProcess::(_processLogs) (92306) Got request for method 'glob' >Thread-18::DEBUG::2014-10-04 20:42:46,286::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 20:42:46,286::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109932::WARNING::2014-10-04 20:43:46,286::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 20:43:46,286::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 20:43:46,286::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 20:43:56,288::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 20:43:56,288::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109933::WARNING::2014-10-04 20:44:56,287::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 20:44:56,288::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 20:44:56,288::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 20:45:06,289::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 20:45:06,290::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 20:45:06,290::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109934::WARNING::2014-10-04 20:46:06,289::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 20:46:06,290::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 20:46:06,290::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 20:46:16,291::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 20:46:16,292::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109935::WARNING::2014-10-04 20:47:16,291::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 20:47:16,292::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 20:47:16,292::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 20:47:26,293::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 20:47:26,294::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109936::WARNING::2014-10-04 20:48:26,293::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 20:48:26,294::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 20:48:26,294::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 20:48:36,295::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 20:48:36,295::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109937::WARNING::2014-10-04 20:49:36,295::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 20:49:36,296::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 20:49:36,296::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 20:49:46,297::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 20:49:46,298::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109938::WARNING::2014-10-04 20:50:46,297::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 20:50:46,298::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 20:50:46,298::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 20:50:56,299::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 20:50:56,300::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 20:50:56,300::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109939::WARNING::2014-10-04 20:51:56,299::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 20:51:56,300::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 20:51:56,300::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 20:52:06,301::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 20:52:06,302::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-04 20:52:41,180::__init__::232::IOProcess::(_processLogs) (92247) Queuing response >Thread-18::DEBUG::2014-10-04 20:52:41,181::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 20:52:41,181::__init__::232::IOProcess::(_processLogs) (92307) Got request for method 'glob' >Thread-109940::WARNING::2014-10-04 20:53:06,301::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 20:53:06,302::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 20:53:06,302::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 20:53:16,303::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 20:53:16,304::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109941::WARNING::2014-10-04 20:54:16,303::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 20:54:16,304::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 20:54:16,304::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 20:54:26,305::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 20:54:26,306::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109942::WARNING::2014-10-04 20:55:26,305::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 20:55:26,306::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 20:55:26,306::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 20:55:36,307::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 20:55:36,307::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109943::WARNING::2014-10-04 20:56:36,307::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 20:56:36,308::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 20:56:36,308::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 20:56:46,308::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 20:56:46,309::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 20:56:46,309::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109944::WARNING::2014-10-04 20:57:46,309::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 20:57:46,310::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 20:57:46,310::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 20:57:56,312::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 20:57:56,312::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109945::WARNING::2014-10-04 20:58:56,312::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 20:58:56,312::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 20:58:56,313::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 20:59:06,314::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 20:59:06,314::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109946::WARNING::2014-10-04 21:00:06,314::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 21:00:06,314::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 21:00:06,314::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 21:00:16,316::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 21:00:16,316::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109947::WARNING::2014-10-04 21:01:16,316::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 21:01:16,316::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 21:01:16,316::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 21:01:26,317::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 21:01:26,318::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109948::WARNING::2014-10-04 21:02:26,317::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 21:02:26,318::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 21:02:26,318::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 21:02:36,318::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 21:02:36,319::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 21:02:36,320::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-04 21:02:45,340::__init__::232::IOProcess::(_processLogs) (92248) Queuing response >Thread-18::DEBUG::2014-10-04 21:02:45,341::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 21:02:45,341::__init__::232::IOProcess::(_processLogs) (92308) Got request for method 'glob' >Thread-18::DEBUG::2014-10-04 21:02:45,341::__init__::232::IOProcess::(_processLogs) (92249) Queuing response >Thread-18::DEBUG::2014-10-04 21:02:45,341::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 21:02:45,341::__init__::232::IOProcess::(_processLogs) (92309) Got request for method 'glob' >Thread-109949::WARNING::2014-10-04 21:03:36,319::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 21:03:36,320::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 21:03:36,320::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 21:03:46,321::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 21:03:46,321::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109950::WARNING::2014-10-04 21:04:46,321::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 21:04:46,322::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 21:04:46,322::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 21:04:56,323::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 21:04:56,323::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109951::WARNING::2014-10-04 21:05:56,323::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 21:05:56,323::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 21:05:56,324::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 21:06:06,325::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 21:06:06,325::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109952::WARNING::2014-10-04 21:07:06,325::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 21:07:06,325::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 21:07:06,326::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 21:07:16,327::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 21:07:16,327::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109953::WARNING::2014-10-04 21:08:16,327::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 21:08:16,327::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 21:08:16,327::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 21:08:26,328::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 21:08:26,329::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 21:08:26,329::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109954::WARNING::2014-10-04 21:09:26,329::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 21:09:26,329::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 21:09:26,329::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 21:09:36,331::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 21:09:36,331::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109955::WARNING::2014-10-04 21:10:36,331::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 21:10:36,331::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 21:10:36,331::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 21:10:46,332::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 21:10:46,333::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109956::WARNING::2014-10-04 21:11:46,332::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 21:11:46,333::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 21:11:46,333::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 21:11:56,334::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 21:11:56,335::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-04 21:12:49,500::__init__::232::IOProcess::(_processLogs) (92250) Queuing response >Thread-18::DEBUG::2014-10-04 21:12:49,501::__init__::232::IOProcess::(_processLogs) (92252) Queuing response >Thread-18::DEBUG::2014-10-04 21:12:49,501::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 21:12:49,501::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 21:12:49,501::__init__::232::IOProcess::(_processLogs) (92311) Got request for method 'glob' >Thread-18::DEBUG::2014-10-04 21:12:49,501::__init__::232::IOProcess::(_processLogs) (92310) Got request for method 'glob' >Thread-109957::WARNING::2014-10-04 21:12:56,334::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 21:12:56,335::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 21:12:56,335::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 21:13:06,336::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 21:13:06,337::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109958::WARNING::2014-10-04 21:14:06,336::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 21:14:06,337::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 21:14:06,337::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 21:14:16,337::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 21:14:16,338::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 21:14:16,339::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109959::WARNING::2014-10-04 21:15:16,338::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 21:15:16,339::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 21:15:16,339::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 21:15:26,340::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 21:15:26,340::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109960::WARNING::2014-10-04 21:16:26,340::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 21:16:26,341::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 21:16:26,341::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 21:16:36,342::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 21:16:36,342::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109961::WARNING::2014-10-04 21:17:36,342::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 21:17:36,343::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 21:17:36,343::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 21:17:46,344::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 21:17:46,344::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109962::WARNING::2014-10-04 21:18:46,344::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 21:18:46,344::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 21:18:46,345::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 21:18:56,346::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 21:18:56,346::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109963::WARNING::2014-10-04 21:19:56,346::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 21:19:56,346::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 21:19:56,347::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 21:20:06,347::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 21:20:06,348::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 21:20:06,348::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109964::WARNING::2014-10-04 21:21:06,348::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 21:21:06,348::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 21:21:06,349::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 21:21:16,350::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 21:21:16,350::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109965::WARNING::2014-10-04 21:22:16,350::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 21:22:16,350::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 21:22:16,351::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 21:22:26,352::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 21:22:26,352::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-04 21:22:55,708::__init__::232::IOProcess::(_processLogs) (92251) Queuing response >Thread-18::DEBUG::2014-10-04 21:22:55,709::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 21:22:55,709::__init__::232::IOProcess::(_processLogs) (92312) Got request for method 'glob' >Thread-18::DEBUG::2014-10-04 21:22:55,709::__init__::232::IOProcess::(_processLogs) (92253) Queuing response >Thread-18::DEBUG::2014-10-04 21:22:55,709::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 21:22:55,709::__init__::232::IOProcess::(_processLogs) (92313) Got request for method 'glob' >Thread-109966::WARNING::2014-10-04 21:23:26,352::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 21:23:26,352::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 21:23:26,353::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 21:23:36,354::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 21:23:36,354::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109967::WARNING::2014-10-04 21:24:36,354::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 21:24:36,354::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 21:24:36,355::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 21:24:46,356::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 21:24:46,356::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109968::WARNING::2014-10-04 21:25:46,356::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 21:25:46,356::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 21:25:46,356::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 21:25:56,357::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 21:25:56,358::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 21:25:56,358::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109969::WARNING::2014-10-04 21:26:56,358::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 21:26:56,358::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 21:26:56,358::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 21:27:06,360::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 21:27:06,360::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109970::WARNING::2014-10-04 21:28:06,359::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 21:28:06,360::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 21:28:06,360::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 21:28:16,361::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 21:28:16,362::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109971::WARNING::2014-10-04 21:29:16,361::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 21:29:16,362::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 21:29:16,362::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 21:29:26,363::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 21:29:26,363::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109972::WARNING::2014-10-04 21:30:26,363::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 21:30:26,363::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 21:30:26,364::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 21:30:36,365::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 21:30:36,365::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109973::WARNING::2014-10-04 21:31:36,365::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 21:31:36,365::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 21:31:36,365::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 21:31:46,366::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 21:31:46,367::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 21:31:46,367::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109974::WARNING::2014-10-04 21:32:46,367::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 21:32:46,367::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 21:32:46,367::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 21:32:56,368::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 21:32:56,369::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-04 21:33:01,916::__init__::232::IOProcess::(_processLogs) (92254) Queuing response >Thread-18::DEBUG::2014-10-04 21:33:01,917::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 21:33:01,917::__init__::232::IOProcess::(_processLogs) (92314) Got request for method 'glob' >Thread-18::DEBUG::2014-10-04 21:33:01,917::__init__::232::IOProcess::(_processLogs) (92255) Queuing response >Thread-18::DEBUG::2014-10-04 21:33:01,918::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 21:33:01,918::__init__::232::IOProcess::(_processLogs) (92315) Got request for method 'glob' >Thread-109975::WARNING::2014-10-04 21:33:56,368::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 21:33:56,369::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 21:33:56,369::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 21:34:06,370::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 21:34:06,370::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109976::WARNING::2014-10-04 21:35:06,370::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 21:35:06,370::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 21:35:06,371::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 21:35:16,372::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 21:35:16,372::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109977::WARNING::2014-10-04 21:36:16,372::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 21:36:16,372::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 21:36:16,372::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 21:36:26,374::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 21:36:26,374::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109978::WARNING::2014-10-04 21:37:26,373::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 21:37:26,374::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 21:37:26,374::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 21:37:36,375::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 21:37:36,375::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 21:37:36,376::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109979::WARNING::2014-10-04 21:38:36,375::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 21:38:36,376::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 21:38:36,376::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 21:38:46,377::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 21:38:46,378::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109980::WARNING::2014-10-04 21:39:46,377::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 21:39:46,378::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 21:39:46,378::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 21:39:56,379::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 21:39:56,379::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109981::WARNING::2014-10-04 21:40:56,379::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 21:40:56,379::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 21:40:56,379::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 21:41:06,381::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 21:41:06,381::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109982::WARNING::2014-10-04 21:42:06,381::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 21:42:06,381::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 21:42:06,381::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 21:42:16,382::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 21:42:16,383::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-04 21:43:06,076::__init__::232::IOProcess::(_processLogs) (92256) Queuing response >Thread-18::DEBUG::2014-10-04 21:43:06,077::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 21:43:06,077::__init__::232::IOProcess::(_processLogs) (92316) Got request for method 'glob' >Thread-18::DEBUG::2014-10-04 21:43:06,077::__init__::232::IOProcess::(_processLogs) (92257) Queuing response >Thread-18::DEBUG::2014-10-04 21:43:06,077::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 21:43:06,077::__init__::232::IOProcess::(_processLogs) (92317) Got request for method 'glob' >Thread-109983::WARNING::2014-10-04 21:43:16,382::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 21:43:16,383::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 21:43:16,383::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 21:43:26,383::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 21:43:26,384::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 21:43:26,384::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109984::WARNING::2014-10-04 21:44:26,384::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 21:44:26,385::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 21:44:26,385::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 21:44:36,386::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 21:44:36,386::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109985::WARNING::2014-10-04 21:45:36,386::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 21:45:36,386::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 21:45:36,387::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 21:45:46,388::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 21:45:46,388::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109986::WARNING::2014-10-04 21:46:46,388::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 21:46:46,388::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 21:46:46,388::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 21:46:56,389::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 21:46:56,390::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109987::WARNING::2014-10-04 21:47:56,389::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 21:47:56,390::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 21:47:56,390::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 21:48:06,391::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 21:48:06,391::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109988::WARNING::2014-10-04 21:49:06,391::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 21:49:06,392::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 21:49:06,392::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 21:49:16,392::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 21:49:16,393::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 21:49:16,393::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109989::WARNING::2014-10-04 21:50:16,393::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 21:50:16,394::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 21:50:16,394::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 21:50:26,395::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 21:50:26,395::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109990::WARNING::2014-10-04 21:51:26,395::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 21:51:26,395::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 21:51:26,395::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 21:51:36,397::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 21:51:36,397::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109991::WARNING::2014-10-04 21:52:36,397::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 21:52:36,397::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 21:52:36,398::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 21:52:46,399::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 21:52:46,400::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-04 21:53:12,284::__init__::232::IOProcess::(_processLogs) (92258) Queuing response >Thread-18::DEBUG::2014-10-04 21:53:12,285::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 21:53:12,285::__init__::232::IOProcess::(_processLogs) (92318) Got request for method 'glob' >Thread-18::DEBUG::2014-10-04 21:53:12,285::__init__::232::IOProcess::(_processLogs) (92259) Queuing response >Thread-18::DEBUG::2014-10-04 21:53:12,285::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 21:53:12,285::__init__::232::IOProcess::(_processLogs) (92319) Got request for method 'glob' >Thread-109992::WARNING::2014-10-04 21:53:46,399::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 21:53:46,400::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 21:53:46,400::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 21:53:56,401::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 21:53:56,401::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109993::WARNING::2014-10-04 21:54:56,401::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 21:54:56,401::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 21:54:56,402::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 21:55:06,402::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 21:55:06,403::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 21:55:06,403::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109994::WARNING::2014-10-04 21:56:06,403::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 21:56:06,403::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 21:56:06,404::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 21:56:16,405::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 21:56:16,405::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109995::WARNING::2014-10-04 21:57:16,405::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 21:57:16,405::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 21:57:16,406::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 21:57:26,407::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 21:57:26,407::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109996::WARNING::2014-10-04 21:58:26,407::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 21:58:26,407::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 21:58:26,407::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 21:58:36,409::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 21:58:36,409::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109997::WARNING::2014-10-04 21:59:36,408::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 21:59:36,409::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 21:59:36,409::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 21:59:46,410::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 21:59:46,411::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109998::WARNING::2014-10-04 22:00:46,410::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 22:00:46,411::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 22:00:46,411::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 22:00:56,412::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 22:00:56,413::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 22:00:56,413::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-109999::WARNING::2014-10-04 22:01:56,412::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 22:01:56,413::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 22:01:56,413::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 22:02:06,414::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 22:02:06,415::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110000::WARNING::2014-10-04 22:03:06,414::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 22:03:06,415::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 22:03:06,415::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 22:03:16,416::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 22:03:16,416::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-04 22:03:18,492::__init__::232::IOProcess::(_processLogs) (92260) Queuing response >Thread-18::DEBUG::2014-10-04 22:03:18,493::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 22:03:18,493::__init__::232::IOProcess::(_processLogs) (92320) Got request for method 'glob' >Thread-18::DEBUG::2014-10-04 22:03:18,493::__init__::232::IOProcess::(_processLogs) (92261) Queuing response >Thread-18::DEBUG::2014-10-04 22:03:18,493::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 22:03:18,493::__init__::232::IOProcess::(_processLogs) (92321) Got request for method 'glob' >Thread-110001::WARNING::2014-10-04 22:04:16,416::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 22:04:16,417::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 22:04:16,417::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 22:04:26,418::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 22:04:26,418::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110002::WARNING::2014-10-04 22:05:26,418::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 22:05:26,418::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 22:05:26,419::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 22:05:36,420::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 22:05:36,420::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110003::WARNING::2014-10-04 22:06:36,420::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 22:06:36,420::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 22:06:36,421::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 22:06:46,421::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 22:06:46,422::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 22:06:46,422::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110004::WARNING::2014-10-04 22:07:46,422::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 22:07:46,422::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 22:07:46,423::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 22:07:56,424::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 22:07:56,424::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110005::WARNING::2014-10-04 22:08:56,424::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 22:08:56,424::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 22:08:56,424::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 22:09:06,426::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 22:09:06,426::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110006::WARNING::2014-10-04 22:10:06,426::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 22:10:06,426::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 22:10:06,426::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 22:10:16,428::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 22:10:16,428::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110007::WARNING::2014-10-04 22:11:16,428::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 22:11:16,428::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 22:11:16,428::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 22:11:26,429::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 22:11:26,430::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110008::WARNING::2014-10-04 22:12:26,429::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 22:12:26,430::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 22:12:26,430::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 22:12:36,430::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 22:12:36,432::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 22:12:36,432::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-04 22:13:22,652::__init__::232::IOProcess::(_processLogs) (92262) Queuing response >Thread-18::DEBUG::2014-10-04 22:13:22,653::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 22:13:22,653::__init__::232::IOProcess::(_processLogs) (92322) Got request for method 'glob' >Thread-110009::WARNING::2014-10-04 22:13:36,432::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 22:13:36,433::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 22:13:36,433::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 22:13:46,434::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 22:13:46,434::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110010::WARNING::2014-10-04 22:14:46,434::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 22:14:46,435::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 22:14:46,435::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 22:14:56,436::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 22:14:56,436::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110011::WARNING::2014-10-04 22:15:56,436::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 22:15:56,436::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 22:15:56,437::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 22:16:06,438::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 22:16:06,438::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110012::WARNING::2014-10-04 22:17:06,438::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 22:17:06,438::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 22:17:06,439::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 22:17:16,440::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 22:17:16,440::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110013::WARNING::2014-10-04 22:18:16,440::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 22:18:16,440::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 22:18:16,441::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 22:18:26,441::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 22:18:26,442::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 22:18:26,442::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110014::WARNING::2014-10-04 22:19:26,442::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 22:19:26,442::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 22:19:26,443::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 22:19:36,444::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 22:19:36,444::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110015::WARNING::2014-10-04 22:20:36,444::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 22:20:36,444::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 22:20:36,444::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 22:20:46,446::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 22:20:46,447::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110016::WARNING::2014-10-04 22:21:46,446::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 22:21:46,447::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 22:21:46,447::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 22:21:56,448::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 22:21:56,449::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110017::WARNING::2014-10-04 22:22:56,448::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 22:22:56,449::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 22:22:56,449::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 22:23:06,450::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 22:23:06,451::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-04 22:23:28,860::__init__::232::IOProcess::(_processLogs) (92263) Queuing response >Thread-18::DEBUG::2014-10-04 22:23:28,861::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 22:23:28,861::__init__::232::IOProcess::(_processLogs) (92323) Got request for method 'glob' >Thread-110018::WARNING::2014-10-04 22:24:06,450::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 22:24:06,451::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 22:24:06,451::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 22:24:16,451::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 22:24:16,452::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 22:24:16,453::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110019::WARNING::2014-10-04 22:25:16,452::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 22:25:16,453::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 22:25:16,453::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 22:25:26,454::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 22:25:26,454::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110020::WARNING::2014-10-04 22:26:26,454::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 22:26:26,455::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 22:26:26,455::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 22:26:36,456::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 22:26:36,456::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110021::WARNING::2014-10-04 22:27:36,456::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 22:27:36,456::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 22:27:36,457::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 22:27:46,458::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 22:27:46,458::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110022::WARNING::2014-10-04 22:28:46,458::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 22:28:46,458::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 22:28:46,458::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 22:28:56,459::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 22:28:56,460::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110023::WARNING::2014-10-04 22:29:56,459::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 22:29:56,460::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 22:29:56,460::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 22:30:06,460::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 22:30:06,461::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 22:30:06,461::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110024::WARNING::2014-10-04 22:31:06,461::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 22:31:06,462::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 22:31:06,462::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 22:31:16,463::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 22:31:16,463::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110025::WARNING::2014-10-04 22:32:16,463::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 22:32:16,463::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 22:32:16,464::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 22:32:26,465::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 22:32:26,465::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110026::WARNING::2014-10-04 22:33:26,465::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 22:33:26,465::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 22:33:26,465::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 22:33:36,467::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 22:33:36,467::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110027::WARNING::2014-10-04 22:34:36,466::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 22:34:36,467::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 22:34:36,467::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 22:34:46,468::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 22:34:46,468::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110028::WARNING::2014-10-04 22:35:46,468::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 22:35:46,469::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 22:35:46,469::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 22:35:56,469::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 22:35:56,470::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 22:35:56,470::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110029::WARNING::2014-10-04 22:36:56,470::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 22:36:56,471::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 22:36:56,471::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 22:37:06,472::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 22:37:06,472::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110030::WARNING::2014-10-04 22:38:06,472::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 22:38:06,472::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 22:38:06,472::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 22:38:16,474::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 22:38:16,474::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110031::WARNING::2014-10-04 22:39:16,474::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 22:39:16,474::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 22:39:16,474::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 22:39:26,475::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 22:39:26,476::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110032::WARNING::2014-10-04 22:40:26,475::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 22:40:26,476::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 22:40:26,476::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 22:40:36,477::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 22:40:36,477::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110033::WARNING::2014-10-04 22:41:36,477::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 22:41:36,477::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 22:41:36,478::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 22:41:46,478::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 22:41:46,479::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 22:41:46,479::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110034::WARNING::2014-10-04 22:42:46,479::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 22:42:46,479::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 22:42:46,480::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 22:42:56,481::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 22:42:56,481::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110035::WARNING::2014-10-04 22:43:56,481::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 22:43:56,481::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 22:43:56,481::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 22:44:06,483::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 22:44:06,483::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110036::WARNING::2014-10-04 22:45:06,483::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 22:45:06,483::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 22:45:06,483::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 22:45:16,485::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 22:45:16,485::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110037::WARNING::2014-10-04 22:46:16,485::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 22:46:16,485::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 22:46:16,485::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 22:46:26,486::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 22:46:26,487::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110038::WARNING::2014-10-04 22:47:26,486::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 22:47:26,487::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 22:47:26,487::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 22:47:36,488::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 22:47:36,489::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 22:47:36,489::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110039::WARNING::2014-10-04 22:48:36,489::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 22:48:36,489::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 22:48:36,489::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 22:48:46,490::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 22:48:46,491::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110040::WARNING::2014-10-04 22:49:46,490::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 22:49:46,491::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 22:49:46,491::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 22:49:56,492::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 22:49:56,493::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110041::WARNING::2014-10-04 22:50:56,492::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 22:50:56,493::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 22:50:56,493::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 22:51:06,494::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 22:51:06,494::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110042::WARNING::2014-10-04 22:52:06,494::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 22:52:06,495::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 22:52:06,495::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 22:52:16,496::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 22:52:16,496::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110043::WARNING::2014-10-04 22:53:16,496::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 22:53:16,497::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 22:53:16,497::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 22:53:26,497::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 22:53:26,498::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 22:53:26,499::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110044::WARNING::2014-10-04 22:54:26,498::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 22:54:26,499::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 22:54:26,499::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 22:54:36,500::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 22:54:36,500::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110045::WARNING::2014-10-04 22:55:36,500::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 22:55:36,501::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 22:55:36,501::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 22:55:46,502::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 22:55:46,502::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110046::WARNING::2014-10-04 22:56:46,502::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 22:56:46,503::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 22:56:46,503::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 22:56:56,504::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 22:56:56,504::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110047::WARNING::2014-10-04 22:57:56,504::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 22:57:56,505::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 22:57:56,505::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 22:58:06,506::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 22:58:06,506::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110048::WARNING::2014-10-04 22:59:06,506::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 22:59:06,506::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 22:59:06,507::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 22:59:16,507::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 22:59:16,509::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 22:59:16,509::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110049::WARNING::2014-10-04 23:00:16,509::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 23:00:16,509::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 23:00:16,510::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 23:00:26,511::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 23:00:26,511::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110050::WARNING::2014-10-04 23:01:26,511::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 23:01:26,511::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 23:01:26,512::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 23:01:36,513::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 23:01:36,513::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110051::WARNING::2014-10-04 23:02:36,513::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 23:02:36,513::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 23:02:36,513::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 23:02:46,515::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 23:02:46,515::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110052::WARNING::2014-10-04 23:03:46,515::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 23:03:46,515::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 23:03:46,515::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 23:03:56,516::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 23:03:56,517::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110053::WARNING::2014-10-04 23:04:56,516::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 23:04:56,517::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 23:04:56,517::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 23:05:06,518::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 23:05:06,519::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 23:05:06,519::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110054::WARNING::2014-10-04 23:06:06,518::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 23:06:06,519::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 23:06:06,519::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 23:06:16,520::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 23:06:16,521::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110055::WARNING::2014-10-04 23:07:16,520::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 23:07:16,521::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 23:07:16,521::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 23:07:26,522::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 23:07:26,523::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110056::WARNING::2014-10-04 23:08:26,522::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 23:08:26,523::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 23:08:26,523::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 23:08:36,524::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 23:08:36,524::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110057::WARNING::2014-10-04 23:09:36,524::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 23:09:36,525::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 23:09:36,525::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 23:09:46,526::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 23:09:46,526::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110058::WARNING::2014-10-04 23:10:46,526::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 23:10:46,527::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 23:10:46,527::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 23:10:56,527::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 23:10:56,528::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 23:10:56,528::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110059::WARNING::2014-10-04 23:11:56,528::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 23:11:56,529::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 23:11:56,529::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 23:12:06,530::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 23:12:06,530::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110060::WARNING::2014-10-04 23:13:06,530::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 23:13:06,531::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 23:13:06,531::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 23:13:16,532::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 23:13:16,532::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110061::WARNING::2014-10-04 23:14:16,532::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 23:14:16,532::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 23:14:16,533::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 23:14:26,534::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 23:14:26,534::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110062::WARNING::2014-10-04 23:15:26,534::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 23:15:26,534::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 23:15:26,535::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 23:15:36,536::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 23:15:36,536::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110063::WARNING::2014-10-04 23:16:36,536::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 23:16:36,536::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 23:16:36,537::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 23:16:46,537::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 23:16:46,538::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 23:16:46,538::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110064::WARNING::2014-10-04 23:17:46,538::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 23:17:46,538::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 23:17:46,539::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 23:17:56,540::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 23:17:56,540::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110065::WARNING::2014-10-04 23:18:56,540::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 23:18:56,540::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 23:18:56,541::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 23:19:06,542::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 23:19:06,542::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110066::WARNING::2014-10-04 23:20:06,542::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 23:20:06,542::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 23:20:06,543::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 23:20:16,544::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 23:20:16,544::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110067::WARNING::2014-10-04 23:21:16,544::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 23:21:16,544::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 23:21:16,545::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 23:21:26,546::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 23:21:26,546::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110068::WARNING::2014-10-04 23:22:26,546::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 23:22:26,546::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 23:22:26,546::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 23:22:36,547::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 23:22:36,548::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 23:22:36,548::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110069::WARNING::2014-10-04 23:23:36,548::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 23:23:36,548::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 23:23:36,549::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 23:23:46,550::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 23:23:46,550::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110070::WARNING::2014-10-04 23:24:46,550::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 23:24:46,550::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 23:24:46,550::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 23:24:56,551::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 23:24:56,552::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110071::WARNING::2014-10-04 23:25:56,551::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 23:25:56,552::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 23:25:56,552::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 23:26:06,553::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 23:26:06,553::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110072::WARNING::2014-10-04 23:27:06,553::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 23:27:06,554::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 23:27:06,554::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 23:27:16,555::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 23:27:16,555::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110073::WARNING::2014-10-04 23:28:16,555::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 23:28:16,555::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 23:28:16,555::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 23:28:26,556::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 23:28:26,557::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 23:28:26,557::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110074::WARNING::2014-10-04 23:29:26,557::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 23:29:26,557::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 23:29:26,557::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 23:29:36,559::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 23:29:36,559::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110075::WARNING::2014-10-04 23:30:36,558::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 23:30:36,559::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 23:30:36,559::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 23:30:46,560::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 23:30:46,561::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110076::WARNING::2014-10-04 23:31:46,560::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 23:31:46,561::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 23:31:46,561::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 23:31:56,562::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 23:31:56,562::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110077::WARNING::2014-10-04 23:32:56,562::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 23:32:56,562::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 23:32:56,563::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 23:33:06,564::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 23:33:06,564::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110078::WARNING::2014-10-04 23:34:06,564::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 23:34:06,564::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 23:34:06,564::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 23:34:16,565::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 23:34:16,566::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 23:34:16,566::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110079::WARNING::2014-10-04 23:35:16,566::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 23:35:16,566::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 23:35:16,566::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 23:35:26,568::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 23:35:26,568::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110080::WARNING::2014-10-04 23:36:26,567::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 23:36:26,568::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 23:36:26,568::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 23:36:36,569::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 23:36:36,569::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110081::WARNING::2014-10-04 23:37:36,569::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 23:37:36,570::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 23:37:36,570::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 23:37:46,571::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 23:37:46,571::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110082::WARNING::2014-10-04 23:38:46,571::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 23:38:46,571::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 23:38:46,572::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 23:38:56,573::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 23:38:56,573::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110083::WARNING::2014-10-04 23:39:56,573::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 23:39:56,573::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 23:39:56,573::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 23:40:06,574::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 23:40:06,575::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 23:40:06,575::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110084::WARNING::2014-10-04 23:41:06,575::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 23:41:06,575::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 23:41:06,575::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 23:41:16,577::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 23:41:16,577::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110085::WARNING::2014-10-04 23:42:16,576::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 23:42:16,577::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 23:42:16,577::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 23:42:26,578::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 23:42:26,579::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110086::WARNING::2014-10-04 23:43:26,578::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 23:43:26,579::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 23:43:26,579::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 23:43:36,580::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 23:43:36,580::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-04 23:44:02,140::__init__::232::IOProcess::(_processLogs) (92264) Queuing response >Thread-18::DEBUG::2014-10-04 23:44:02,141::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-04 23:44:02,141::__init__::232::IOProcess::(_processLogs) (92324) Got request for method 'glob' >Thread-110087::WARNING::2014-10-04 23:44:36,580::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 23:44:36,581::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 23:44:36,581::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 23:44:46,582::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 23:44:46,582::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110088::WARNING::2014-10-04 23:45:46,582::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 23:45:46,583::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 23:45:46,583::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 23:45:56,583::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 23:45:56,584::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 23:45:56,584::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110089::WARNING::2014-10-04 23:46:56,584::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 23:46:56,585::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 23:46:56,585::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 23:47:06,586::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 23:47:06,586::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110090::WARNING::2014-10-04 23:48:06,586::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 23:48:06,586::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 23:48:06,587::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 23:48:16,588::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 23:48:16,588::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110091::WARNING::2014-10-04 23:49:16,588::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 23:49:16,588::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 23:49:16,589::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 23:49:26,590::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 23:49:26,590::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110092::WARNING::2014-10-04 23:50:26,590::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 23:50:26,590::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 23:50:26,590::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 23:50:36,592::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 23:50:36,592::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110093::WARNING::2014-10-04 23:51:36,592::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 23:51:36,592::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 23:51:36,592::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 23:51:46,593::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 23:51:46,594::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 23:51:46,594::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110094::WARNING::2014-10-04 23:52:46,594::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 23:52:46,594::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 23:52:46,595::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 23:52:56,596::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 23:52:56,596::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110095::WARNING::2014-10-04 23:53:56,596::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 23:53:56,596::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 23:53:56,596::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 23:54:06,598::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 23:54:06,598::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110096::WARNING::2014-10-04 23:55:06,598::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 23:55:06,598::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 23:55:06,599::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 23:55:16,600::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 23:55:16,600::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110097::WARNING::2014-10-04 23:56:16,600::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 23:56:16,601::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 23:56:16,601::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 23:56:26,602::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 23:56:26,602::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110098::WARNING::2014-10-04 23:57:26,602::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 23:57:26,602::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 23:57:26,603::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-04 23:57:36,603::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-04 23:57:36,604::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 23:57:36,604::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110099::WARNING::2014-10-04 23:58:36,604::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 23:58:36,605::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 23:58:36,605::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 23:58:46,606::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 23:58:46,606::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110100::WARNING::2014-10-04 23:59:46,606::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-04 23:59:46,606::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-04 23:59:46,607::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-04 23:59:56,608::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-04 23:59:56,608::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110101::WARNING::2014-10-05 00:00:56,608::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 00:00:56,608::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 00:00:56,608::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 00:01:06,610::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 00:01:06,610::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110102::WARNING::2014-10-05 00:02:06,610::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 00:02:06,610::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 00:02:06,610::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 00:02:16,611::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 00:02:16,612::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110103::WARNING::2014-10-05 00:03:16,611::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 00:03:16,612::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 00:03:16,612::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 00:03:26,613::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 00:03:26,614::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 00:03:26,614::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110104::WARNING::2014-10-05 00:04:26,613::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 00:04:26,614::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 00:04:26,614::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 00:04:36,615::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 00:04:36,616::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110105::WARNING::2014-10-05 00:05:36,615::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 00:05:36,616::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 00:05:36,616::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 00:05:46,617::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 00:05:46,617::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110106::WARNING::2014-10-05 00:06:46,617::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 00:06:46,618::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 00:06:46,618::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 00:06:56,619::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 00:06:56,619::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110107::WARNING::2014-10-05 00:07:56,619::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 00:07:56,620::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 00:07:56,620::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 00:08:06,621::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 00:08:06,621::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110108::WARNING::2014-10-05 00:09:06,621::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 00:09:06,621::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 00:09:06,622::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 00:09:16,622::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 00:09:16,623::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 00:09:16,623::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110109::WARNING::2014-10-05 00:10:16,623::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 00:10:16,624::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 00:10:16,624::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 00:10:26,625::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 00:10:26,625::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110110::WARNING::2014-10-05 00:11:26,625::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 00:11:26,625::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 00:11:26,626::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 00:11:36,627::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 00:11:36,627::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110111::WARNING::2014-10-05 00:12:36,627::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 00:12:36,627::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 00:12:36,627::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 00:12:46,629::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 00:12:46,629::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110112::WARNING::2014-10-05 00:13:46,629::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 00:13:46,629::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 00:13:46,629::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 00:13:56,631::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 00:13:56,631::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110113::WARNING::2014-10-05 00:14:56,630::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 00:14:56,631::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 00:14:56,631::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 00:15:06,632::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 00:15:06,633::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 00:15:06,633::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110114::WARNING::2014-10-05 00:16:06,633::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 00:16:06,633::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 00:16:06,633::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 00:16:16,635::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 00:16:16,635::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110115::WARNING::2014-10-05 00:17:16,634::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 00:17:16,635::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 00:17:16,635::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 00:17:26,636::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 00:17:26,637::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110116::WARNING::2014-10-05 00:18:26,636::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 00:18:26,637::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 00:18:26,637::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 00:18:36,638::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 00:18:36,639::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110117::WARNING::2014-10-05 00:19:36,638::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 00:19:36,639::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 00:19:36,639::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 00:19:46,640::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 00:19:46,641::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110118::WARNING::2014-10-05 00:20:46,640::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 00:20:46,641::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 00:20:46,641::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 00:20:56,642::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 00:20:56,643::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 00:20:56,643::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110119::WARNING::2014-10-05 00:21:56,643::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 00:21:56,643::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 00:21:56,644::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 00:22:06,645::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 00:22:06,645::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110120::WARNING::2014-10-05 00:23:06,645::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 00:23:06,645::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 00:23:06,646::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 00:23:16,647::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 00:23:16,647::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110121::WARNING::2014-10-05 00:24:16,647::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 00:24:16,647::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 00:24:16,647::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 00:24:18,780::__init__::232::IOProcess::(_processLogs) (92265) Queuing response >Thread-18::DEBUG::2014-10-05 00:24:18,781::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 00:24:18,781::__init__::232::IOProcess::(_processLogs) (92325) Got request for method 'glob' >Thread-18::DEBUG::2014-10-05 00:24:26,649::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 00:24:26,649::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110122::WARNING::2014-10-05 00:25:26,648::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 00:25:26,649::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 00:25:26,649::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 00:25:36,650::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 00:25:36,651::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110123::WARNING::2014-10-05 00:26:36,650::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 00:26:36,651::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 00:26:36,651::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 00:26:46,651::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 00:26:46,652::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 00:26:46,652::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110124::WARNING::2014-10-05 00:27:46,652::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 00:27:46,653::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 00:27:46,653::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 00:27:56,654::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 00:27:56,654::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110125::WARNING::2014-10-05 00:28:56,654::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 00:28:56,654::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 00:28:56,654::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 00:29:06,656::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 00:29:06,656::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110126::WARNING::2014-10-05 00:30:06,656::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 00:30:06,656::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 00:30:06,658::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 00:30:16,659::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 00:30:16,659::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110127::WARNING::2014-10-05 00:31:16,659::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 00:31:16,660::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 00:31:16,660::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 00:31:26,661::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 00:31:26,661::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110128::WARNING::2014-10-05 00:32:26,661::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 00:32:26,661::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 00:32:26,662::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 00:32:36,662::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 00:32:36,663::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 00:32:36,663::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110129::WARNING::2014-10-05 00:33:36,663::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 00:33:36,663::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 00:33:36,664::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 00:33:46,665::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 00:33:46,665::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-05 00:34:22,940::__init__::232::IOProcess::(_processLogs) (92266) Queuing response >Thread-18::DEBUG::2014-10-05 00:34:22,941::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 00:34:22,941::__init__::232::IOProcess::(_processLogs) (92326) Got request for method 'glob' >Thread-110130::WARNING::2014-10-05 00:34:46,665::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 00:34:46,665::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 00:34:46,665::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 00:34:56,666::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 00:34:56,667::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110131::WARNING::2014-10-05 00:35:56,666::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 00:35:56,667::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 00:35:56,667::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 00:36:06,668::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 00:36:06,668::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110132::WARNING::2014-10-05 00:37:06,668::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 00:37:06,668::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 00:37:06,669::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 00:37:16,670::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 00:37:16,670::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110133::WARNING::2014-10-05 00:38:16,670::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 00:38:16,670::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 00:38:16,671::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 00:38:26,671::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 00:38:26,672::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 00:38:26,672::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110134::WARNING::2014-10-05 00:39:26,672::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 00:39:26,672::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 00:39:26,673::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 00:39:36,674::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 00:39:36,674::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110135::WARNING::2014-10-05 00:40:36,674::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 00:40:36,674::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 00:40:36,674::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 00:40:46,676::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 00:40:46,676::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110136::WARNING::2014-10-05 00:41:46,676::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 00:41:46,676::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 00:41:46,676::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 00:41:56,678::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 00:41:56,678::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110137::WARNING::2014-10-05 00:42:56,677::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 00:42:56,678::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 00:42:56,678::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 00:43:06,679::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 00:43:06,680::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110138::WARNING::2014-10-05 00:44:06,679::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 00:44:06,680::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 00:44:06,680::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 00:44:16,680::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 00:44:16,681::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 00:44:16,681::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-05 00:44:27,100::__init__::232::IOProcess::(_processLogs) (92267) Queuing response >Thread-18::DEBUG::2014-10-05 00:44:27,101::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 00:44:27,101::__init__::232::IOProcess::(_processLogs) (92327) Got request for method 'glob' >Thread-110139::WARNING::2014-10-05 00:45:16,681::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 00:45:16,682::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 00:45:16,682::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 00:45:26,683::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 00:45:26,684::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110140::WARNING::2014-10-05 00:46:26,683::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 00:46:26,684::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 00:46:26,684::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 00:46:36,685::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 00:46:36,685::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110141::WARNING::2014-10-05 00:47:36,685::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 00:47:36,686::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 00:47:36,686::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 00:47:46,687::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 00:47:46,687::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110142::WARNING::2014-10-05 00:48:46,687::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 00:48:46,688::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 00:48:46,688::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 00:48:56,689::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 00:48:56,689::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110143::WARNING::2014-10-05 00:49:56,689::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 00:49:56,690::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 00:49:56,690::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 00:50:06,690::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 00:50:06,691::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 00:50:06,691::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110144::WARNING::2014-10-05 00:51:06,691::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 00:51:06,692::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 00:51:06,692::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 00:51:16,693::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 00:51:16,693::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110145::WARNING::2014-10-05 00:52:16,693::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 00:52:16,694::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 00:52:16,694::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 00:52:26,695::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 00:52:26,696::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110146::WARNING::2014-10-05 00:53:26,695::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 00:53:26,696::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 00:53:26,696::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 00:53:36,697::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 00:53:36,697::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110147::WARNING::2014-10-05 00:54:36,697::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 00:54:36,698::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 00:54:36,698::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 00:54:46,699::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 00:54:46,700::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110148::WARNING::2014-10-05 00:55:46,699::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 00:55:46,700::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 00:55:46,700::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 00:55:56,700::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 00:55:56,701::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 00:55:56,702::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110149::WARNING::2014-10-05 00:56:56,701::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 00:56:56,702::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 00:56:56,702::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 00:57:06,703::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 00:57:06,703::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110150::WARNING::2014-10-05 00:58:06,703::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 00:58:06,704::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 00:58:06,704::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 00:58:16,705::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 00:58:16,705::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110151::WARNING::2014-10-05 00:59:16,705::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 00:59:16,706::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 00:59:16,706::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 00:59:26,707::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 00:59:26,707::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110152::WARNING::2014-10-05 01:00:26,707::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 01:00:26,707::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 01:00:26,708::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 01:00:36,709::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 01:00:36,709::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110153::WARNING::2014-10-05 01:01:36,709::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 01:01:36,709::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 01:01:36,710::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 01:01:46,710::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 01:01:46,711::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 01:01:46,711::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110154::WARNING::2014-10-05 01:02:46,711::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 01:02:46,711::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 01:02:46,712::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 01:02:56,713::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 01:02:56,713::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110155::WARNING::2014-10-05 01:03:56,713::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 01:03:56,713::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 01:03:56,713::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 01:04:06,715::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 01:04:06,715::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110156::WARNING::2014-10-05 01:05:06,714::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 01:05:06,715::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 01:05:06,715::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 01:05:16,716::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 01:05:16,717::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110157::WARNING::2014-10-05 01:06:16,716::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 01:06:16,717::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 01:06:16,717::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 01:06:26,718::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 01:06:26,718::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110158::WARNING::2014-10-05 01:07:26,718::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 01:07:26,719::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 01:07:26,719::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 01:07:36,719::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 01:07:36,720::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 01:07:36,720::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110159::WARNING::2014-10-05 01:08:36,720::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 01:08:36,721::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 01:08:36,721::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 01:08:46,722::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 01:08:46,722::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110160::WARNING::2014-10-05 01:09:46,722::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 01:09:46,722::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 01:09:46,723::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 01:09:56,724::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 01:09:56,724::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110161::WARNING::2014-10-05 01:10:56,724::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 01:10:56,724::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 01:10:56,724::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 01:11:06,726::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 01:11:06,726::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110162::WARNING::2014-10-05 01:12:06,725::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 01:12:06,726::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 01:12:06,726::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 01:12:16,728::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 01:12:16,728::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110163::WARNING::2014-10-05 01:13:16,727::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 01:13:16,728::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 01:13:16,728::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 01:13:26,729::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 01:13:26,730::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 01:13:26,730::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110164::WARNING::2014-10-05 01:14:26,730::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 01:14:26,730::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 01:14:26,730::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 01:14:36,732::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 01:14:36,732::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110165::WARNING::2014-10-05 01:15:36,732::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 01:15:36,733::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 01:15:36,733::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 01:15:46,734::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 01:15:46,734::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110166::WARNING::2014-10-05 01:16:46,734::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 01:16:46,735::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 01:16:46,735::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 01:16:56,736::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 01:16:56,736::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110167::WARNING::2014-10-05 01:17:56,736::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 01:17:56,737::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 01:17:56,737::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 01:18:06,738::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 01:18:06,738::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110168::WARNING::2014-10-05 01:19:06,738::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 01:19:06,739::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 01:19:06,739::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 01:19:16,739::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 01:19:16,740::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 01:19:16,740::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110169::WARNING::2014-10-05 01:20:16,740::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 01:20:16,741::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 01:20:16,741::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 01:20:26,742::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 01:20:26,742::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110170::WARNING::2014-10-05 01:21:26,742::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 01:21:26,743::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 01:21:26,743::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 01:21:36,744::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 01:21:36,744::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110171::WARNING::2014-10-05 01:22:36,744::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 01:22:36,745::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 01:22:36,745::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 01:22:46,746::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 01:22:46,746::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110172::WARNING::2014-10-05 01:23:46,746::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 01:23:46,747::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 01:23:46,747::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 01:23:56,748::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 01:23:56,748::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110173::WARNING::2014-10-05 01:24:56,748::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 01:24:56,748::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 01:24:56,749::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 01:25:06,749::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 01:25:06,750::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 01:25:06,750::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110174::WARNING::2014-10-05 01:26:06,750::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 01:26:06,750::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 01:26:06,751::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 01:26:16,752::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 01:26:16,752::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110175::WARNING::2014-10-05 01:27:16,752::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 01:27:16,752::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 01:27:16,752::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 01:27:26,753::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 01:27:26,754::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110176::WARNING::2014-10-05 01:28:26,753::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 01:28:26,754::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 01:28:26,754::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 01:28:36,755::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 01:28:36,755::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110177::WARNING::2014-10-05 01:29:36,755::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 01:29:36,755::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 01:29:36,756::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 01:29:46,757::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 01:29:46,757::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110178::WARNING::2014-10-05 01:30:46,757::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 01:30:46,757::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 01:30:46,757::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 01:30:56,758::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 01:30:56,759::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 01:30:56,759::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110179::WARNING::2014-10-05 01:31:56,759::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 01:31:56,759::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 01:31:56,759::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 01:32:06,760::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 01:32:06,761::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110180::WARNING::2014-10-05 01:33:06,760::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 01:33:06,761::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 01:33:06,761::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 01:33:16,762::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 01:33:16,762::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110181::WARNING::2014-10-05 01:34:16,762::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 01:34:16,763::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 01:34:16,763::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 01:34:26,764::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 01:34:26,764::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-05 01:34:47,901::__init__::232::IOProcess::(_processLogs) (92268) Queuing response >Thread-18::DEBUG::2014-10-05 01:34:47,901::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 01:34:47,901::__init__::232::IOProcess::(_processLogs) (92328) Got request for method 'glob' >Thread-110182::WARNING::2014-10-05 01:35:26,764::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 01:35:26,764::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 01:35:26,765::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 01:35:36,766::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 01:35:36,766::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110183::WARNING::2014-10-05 01:36:36,766::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 01:36:36,766::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 01:36:36,767::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 01:36:46,767::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 01:36:46,768::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 01:36:46,768::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110184::WARNING::2014-10-05 01:37:46,768::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 01:37:46,768::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 01:37:46,769::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 01:37:56,770::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 01:37:56,770::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110185::WARNING::2014-10-05 01:38:56,770::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 01:38:56,770::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 01:38:56,770::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 01:39:06,772::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 01:39:06,772::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110186::WARNING::2014-10-05 01:40:06,772::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 01:40:06,772::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 01:40:06,772::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 01:40:16,774::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 01:40:16,774::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110187::WARNING::2014-10-05 01:41:16,773::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 01:41:16,774::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 01:41:16,774::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 01:41:26,775::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 01:41:26,776::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110188::WARNING::2014-10-05 01:42:26,775::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 01:42:26,776::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 01:42:26,776::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 01:42:36,777::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 01:42:36,778::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 01:42:36,779::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110189::WARNING::2014-10-05 01:43:36,778::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 01:43:36,779::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 01:43:36,779::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 01:43:46,780::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 01:43:46,781::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110190::WARNING::2014-10-05 01:44:46,780::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 01:44:46,781::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 01:44:46,781::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 01:44:56,782::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 01:44:56,782::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110191::WARNING::2014-10-05 01:45:56,782::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 01:45:56,783::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 01:45:56,783::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 01:46:06,784::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 01:46:06,785::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110192::WARNING::2014-10-05 01:47:06,784::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 01:47:06,785::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 01:47:06,785::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 01:47:16,786::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 01:47:16,787::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110193::WARNING::2014-10-05 01:48:16,786::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 01:48:16,787::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 01:48:16,787::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 01:48:26,788::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 01:48:26,789::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 01:48:26,789::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110194::WARNING::2014-10-05 01:49:26,788::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 01:49:26,789::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 01:49:26,789::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 01:49:36,790::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 01:49:36,791::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110195::WARNING::2014-10-05 01:50:36,790::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 01:50:36,791::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 01:50:36,791::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 01:50:46,792::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 01:50:46,793::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110196::WARNING::2014-10-05 01:51:46,792::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 01:51:46,793::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 01:51:46,793::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 01:51:56,795::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 01:51:56,795::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110197::WARNING::2014-10-05 01:52:56,794::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 01:52:56,795::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 01:52:56,795::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 01:53:06,797::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 01:53:06,797::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110198::WARNING::2014-10-05 01:54:06,797::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 01:54:06,797::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 01:54:06,797::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 01:54:16,798::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 01:54:16,799::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 01:54:16,799::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-05 01:54:56,220::__init__::232::IOProcess::(_processLogs) (92269) Queuing response >Thread-18::DEBUG::2014-10-05 01:54:56,221::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 01:54:56,221::__init__::232::IOProcess::(_processLogs) (92329) Got request for method 'glob' >Thread-110199::WARNING::2014-10-05 01:55:16,799::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 01:55:16,799::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 01:55:16,799::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 01:55:26,801::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 01:55:26,801::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110200::WARNING::2014-10-05 01:56:26,800::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 01:56:26,801::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 01:56:26,801::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 01:56:36,802::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 01:56:36,803::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110201::WARNING::2014-10-05 01:57:36,802::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 01:57:36,803::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 01:57:36,803::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 01:57:46,804::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 01:57:46,804::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110202::WARNING::2014-10-05 01:58:46,804::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 01:58:46,804::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 01:58:46,805::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 01:58:56,806::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 01:58:56,806::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110203::WARNING::2014-10-05 01:59:56,806::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 01:59:56,806::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 01:59:56,807::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 02:00:06,807::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 02:00:06,808::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 02:00:06,808::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110204::WARNING::2014-10-05 02:01:06,808::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 02:01:06,808::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 02:01:06,810::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 02:01:16,811::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 02:01:16,811::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110205::WARNING::2014-10-05 02:02:16,811::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 02:02:16,811::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 02:02:16,812::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 02:02:26,813::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 02:02:26,813::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110206::WARNING::2014-10-05 02:03:26,813::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 02:03:26,813::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 02:03:26,814::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 02:03:36,815::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 02:03:36,815::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110207::WARNING::2014-10-05 02:04:36,815::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 02:04:36,815::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 02:04:36,815::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 02:04:46,817::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 02:04:46,817::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110208::WARNING::2014-10-05 02:05:46,817::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 02:05:46,817::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 02:05:46,817::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 02:05:56,818::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 02:05:56,819::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 02:05:56,819::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110209::WARNING::2014-10-05 02:06:56,819::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 02:06:56,819::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 02:06:56,819::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 02:07:06,821::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 02:07:06,821::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110210::WARNING::2014-10-05 02:08:06,821::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 02:08:06,821::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 02:08:06,821::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 02:08:16,823::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 02:08:16,823::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110211::WARNING::2014-10-05 02:09:16,822::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 02:09:16,823::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 02:09:16,823::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 02:09:26,824::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 02:09:26,825::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110212::WARNING::2014-10-05 02:10:26,824::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 02:10:26,825::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 02:10:26,825::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 02:10:36,826::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 02:10:36,826::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110213::WARNING::2014-10-05 02:11:36,826::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 02:11:36,827::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 02:11:36,827::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 02:11:46,827::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 02:11:46,828::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 02:11:46,828::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110214::WARNING::2014-10-05 02:12:46,828::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 02:12:46,829::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 02:12:46,829::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 02:12:56,830::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 02:12:56,830::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110215::WARNING::2014-10-05 02:13:56,830::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 02:13:56,831::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 02:13:56,831::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 02:14:06,832::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 02:14:06,832::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110216::WARNING::2014-10-05 02:15:06,832::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 02:15:06,833::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 02:15:06,833::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 02:15:16,834::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 02:15:16,834::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110217::WARNING::2014-10-05 02:16:16,834::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 02:16:16,835::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 02:16:16,835::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 02:16:26,836::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 02:16:26,836::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110218::WARNING::2014-10-05 02:17:26,836::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 02:17:26,836::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 02:17:26,837::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 02:17:36,837::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 02:17:36,838::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 02:17:36,838::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110219::WARNING::2014-10-05 02:18:36,838::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 02:18:36,839::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 02:18:36,839::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 02:18:46,840::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 02:18:46,840::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110220::WARNING::2014-10-05 02:19:46,840::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 02:19:46,841::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 02:19:46,841::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 02:19:56,842::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 02:19:56,842::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110221::WARNING::2014-10-05 02:20:56,842::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 02:20:56,843::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 02:20:56,843::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 02:21:06,844::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 02:21:06,844::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110222::WARNING::2014-10-05 02:22:06,844::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 02:22:06,845::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 02:22:06,845::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 02:22:16,846::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 02:22:16,846::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110223::WARNING::2014-10-05 02:23:16,846::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 02:23:16,847::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 02:23:16,847::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 02:23:26,847::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 02:23:26,848::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 02:23:26,848::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110224::WARNING::2014-10-05 02:24:26,848::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 02:24:26,849::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 02:24:26,849::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 02:24:36,850::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 02:24:36,851::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110225::WARNING::2014-10-05 02:25:36,850::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 02:25:36,851::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 02:25:36,851::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 02:25:46,852::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 02:25:46,852::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110226::WARNING::2014-10-05 02:26:46,852::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 02:26:46,853::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 02:26:46,853::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 02:26:56,854::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 02:26:56,854::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110227::WARNING::2014-10-05 02:27:56,854::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 02:27:56,854::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 02:27:56,854::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 02:28:06,856::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 02:28:06,856::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110228::WARNING::2014-10-05 02:29:06,856::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 02:29:06,856::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 02:29:06,856::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 02:29:16,857::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 02:29:16,858::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 02:29:16,858::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110229::WARNING::2014-10-05 02:30:16,858::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 02:30:16,858::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 02:30:16,858::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 02:30:26,860::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 02:30:26,860::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110230::WARNING::2014-10-05 02:31:26,860::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 02:31:26,860::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 02:31:26,860::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 02:31:36,862::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 02:31:36,862::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110231::WARNING::2014-10-05 02:32:36,861::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 02:32:36,862::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 02:32:36,862::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 02:32:46,863::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 02:32:46,864::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110232::WARNING::2014-10-05 02:33:46,863::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 02:33:46,864::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 02:33:46,864::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 02:33:56,865::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 02:33:56,865::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110233::WARNING::2014-10-05 02:34:56,865::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 02:34:56,866::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 02:34:56,866::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 02:35:06,866::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 02:35:06,867::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 02:35:06,867::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110234::WARNING::2014-10-05 02:36:06,867::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 02:36:06,868::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 02:36:06,868::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 02:36:16,869::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 02:36:16,869::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110235::WARNING::2014-10-05 02:37:16,869::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 02:37:16,870::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 02:37:16,870::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 02:37:26,871::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 02:37:26,871::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110236::WARNING::2014-10-05 02:38:26,871::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 02:38:26,872::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 02:38:26,872::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 02:38:36,873::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 02:38:36,873::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110237::WARNING::2014-10-05 02:39:36,873::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 02:39:36,873::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 02:39:36,874::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 02:39:46,875::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 02:39:46,875::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110238::WARNING::2014-10-05 02:40:46,875::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 02:40:46,875::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 02:40:46,876::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 02:40:56,876::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 02:40:56,877::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 02:40:56,877::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110239::WARNING::2014-10-05 02:41:56,877::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 02:41:56,877::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 02:41:56,878::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 02:42:06,879::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 02:42:06,879::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110240::WARNING::2014-10-05 02:43:06,879::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 02:43:06,879::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 02:43:06,879::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 02:43:16,881::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 02:43:16,881::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110241::WARNING::2014-10-05 02:44:16,881::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 02:44:16,881::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 02:44:16,881::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 02:44:26,883::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 02:44:26,883::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110242::WARNING::2014-10-05 02:45:26,882::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 02:45:26,883::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 02:45:26,883::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 02:45:36,884::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 02:45:36,885::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110243::WARNING::2014-10-05 02:46:36,884::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 02:46:36,885::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 02:46:36,885::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 02:46:46,886::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 02:46:46,887::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 02:46:46,887::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110244::WARNING::2014-10-05 02:47:46,887::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 02:47:46,887::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 02:47:46,887::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 02:47:56,889::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 02:47:56,889::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110245::WARNING::2014-10-05 02:48:56,889::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 02:48:56,889::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 02:48:56,889::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 02:49:06,891::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 02:49:06,891::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110246::WARNING::2014-10-05 02:50:06,891::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 02:50:06,891::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 02:50:06,892::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 02:50:16,893::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 02:50:16,893::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110247::WARNING::2014-10-05 02:51:16,893::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 02:51:16,893::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 02:51:16,894::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 02:51:26,895::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 02:51:26,895::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110248::WARNING::2014-10-05 02:52:26,895::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 02:52:26,895::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 02:52:26,895::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 02:52:36,896::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 02:52:36,897::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 02:52:36,897::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110249::WARNING::2014-10-05 02:53:36,897::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 02:53:36,897::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 02:53:36,897::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 02:53:46,899::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 02:53:46,899::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110250::WARNING::2014-10-05 02:54:46,899::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 02:54:46,899::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 02:54:46,899::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 02:54:56,900::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 02:54:56,901::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110251::WARNING::2014-10-05 02:55:56,900::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 02:55:56,901::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 02:55:56,901::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 02:56:06,902::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 02:56:06,902::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110252::WARNING::2014-10-05 02:57:06,902::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 02:57:06,902::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 02:57:06,903::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 02:57:16,904::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 02:57:16,904::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110253::WARNING::2014-10-05 02:58:16,904::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 02:58:16,904::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 02:58:16,904::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 02:58:26,905::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 02:58:26,906::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 02:58:26,906::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110254::WARNING::2014-10-05 02:59:26,906::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 02:59:26,906::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 02:59:26,906::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 02:59:36,908::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 02:59:36,908::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110255::WARNING::2014-10-05 03:00:36,907::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 03:00:36,908::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 03:00:36,908::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 03:00:46,909::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 03:00:46,909::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110256::WARNING::2014-10-05 03:01:46,909::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 03:01:46,910::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 03:01:46,910::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 03:01:56,911::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 03:01:56,911::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110257::WARNING::2014-10-05 03:02:56,911::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 03:02:56,911::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 03:02:56,912::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 03:03:06,913::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 03:03:06,913::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110258::WARNING::2014-10-05 03:04:06,913::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 03:04:06,913::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 03:04:06,913::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 03:04:16,914::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 03:04:16,915::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 03:04:16,915::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110259::WARNING::2014-10-05 03:05:16,914::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 03:05:16,915::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 03:05:16,915::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 03:05:25,341::__init__::232::IOProcess::(_processLogs) (92270) Queuing response >Thread-18::DEBUG::2014-10-05 03:05:25,341::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 03:05:25,341::__init__::232::IOProcess::(_processLogs) (92330) Got request for method 'glob' >Thread-18::DEBUG::2014-10-05 03:05:26,916::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 03:05:26,916::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110260::WARNING::2014-10-05 03:06:26,916::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 03:06:26,917::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 03:06:26,917::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 03:06:36,918::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 03:06:36,918::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110261::WARNING::2014-10-05 03:07:36,918::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 03:07:36,918::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 03:07:36,919::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 03:07:46,920::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 03:07:46,920::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110262::WARNING::2014-10-05 03:08:46,920::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 03:08:46,920::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 03:08:46,920::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 03:08:56,922::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 03:08:56,922::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110263::WARNING::2014-10-05 03:09:56,921::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 03:09:56,922::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 03:09:56,922::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 03:10:06,922::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 03:10:06,923::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 03:10:06,924::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110264::WARNING::2014-10-05 03:11:06,923::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 03:11:06,924::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 03:11:06,924::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 03:11:16,925::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 03:11:16,925::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110265::WARNING::2014-10-05 03:12:16,925::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 03:12:16,925::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 03:12:16,926::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 03:12:26,927::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 03:12:26,927::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110266::WARNING::2014-10-05 03:13:26,927::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 03:13:26,927::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 03:13:26,928::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 03:13:36,929::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 03:13:36,929::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110267::WARNING::2014-10-05 03:14:36,929::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 03:14:36,929::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 03:14:36,929::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 03:14:46,930::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 03:14:46,930::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-05 03:15:29,500::__init__::232::IOProcess::(_processLogs) (92271) Queuing response >Thread-18::DEBUG::2014-10-05 03:15:29,501::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 03:15:29,501::__init__::232::IOProcess::(_processLogs) (92331) Got request for method 'glob' >Thread-18::DEBUG::2014-10-05 03:15:29,501::__init__::232::IOProcess::(_processLogs) (92272) Queuing response >Thread-18::DEBUG::2014-10-05 03:15:29,501::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 03:15:29,501::__init__::232::IOProcess::(_processLogs) (92332) Got request for method 'glob' >Thread-110268::WARNING::2014-10-05 03:15:46,930::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 03:15:46,931::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 03:15:46,931::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 03:15:56,931::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 03:15:56,932::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 03:15:56,932::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110269::WARNING::2014-10-05 03:16:56,932::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 03:16:56,933::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 03:16:56,933::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 03:17:06,934::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 03:17:06,934::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110270::WARNING::2014-10-05 03:18:06,934::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 03:18:06,935::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 03:18:06,935::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 03:18:16,936::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 03:18:16,936::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110271::WARNING::2014-10-05 03:19:16,936::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 03:19:16,937::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 03:19:16,937::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 03:19:26,938::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 03:19:26,938::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110272::WARNING::2014-10-05 03:20:26,938::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 03:20:26,939::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 03:20:26,939::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 03:20:36,940::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 03:20:36,940::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110273::WARNING::2014-10-05 03:21:36,940::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 03:21:36,940::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 03:21:36,941::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 03:21:46,941::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 03:21:46,942::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 03:21:46,942::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110274::WARNING::2014-10-05 03:22:46,942::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 03:22:46,943::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 03:22:46,943::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 03:22:56,944::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 03:22:56,944::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110275::WARNING::2014-10-05 03:23:56,944::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 03:23:56,945::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 03:23:56,945::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 03:24:06,946::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 03:24:06,946::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110276::WARNING::2014-10-05 03:25:06,946::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 03:25:06,947::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 03:25:06,948::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 03:25:16,949::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 03:25:16,949::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-05 03:25:33,660::__init__::232::IOProcess::(_processLogs) (92273) Queuing response >Thread-18::DEBUG::2014-10-05 03:25:33,661::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 03:25:33,661::__init__::232::IOProcess::(_processLogs) (92333) Got request for method 'glob' >Thread-18::DEBUG::2014-10-05 03:25:33,661::__init__::232::IOProcess::(_processLogs) (92274) Queuing response >Thread-18::DEBUG::2014-10-05 03:25:33,661::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 03:25:33,661::__init__::232::IOProcess::(_processLogs) (92334) Got request for method 'glob' >Thread-110277::WARNING::2014-10-05 03:26:16,949::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 03:26:16,949::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 03:26:16,950::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 03:26:26,951::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 03:26:26,951::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110278::WARNING::2014-10-05 03:27:26,951::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 03:27:26,951::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 03:27:26,951::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 03:27:36,952::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 03:27:36,953::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 03:27:36,953::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110279::WARNING::2014-10-05 03:28:36,953::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 03:28:36,953::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 03:28:36,953::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 03:28:46,955::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 03:28:46,955::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110280::WARNING::2014-10-05 03:29:46,955::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 03:29:46,955::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 03:29:46,955::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 03:29:56,957::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 03:29:56,957::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110281::WARNING::2014-10-05 03:30:56,956::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 03:30:56,957::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 03:30:56,957::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 03:31:06,958::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 03:31:06,959::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110282::WARNING::2014-10-05 03:32:06,958::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 03:32:06,959::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 03:32:06,959::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 03:32:16,960::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 03:32:16,960::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110283::WARNING::2014-10-05 03:33:16,960::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 03:33:16,961::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 03:33:16,961::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 03:33:26,961::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 03:33:26,962::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 03:33:26,962::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110284::WARNING::2014-10-05 03:34:26,962::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 03:34:26,963::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 03:34:26,963::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 03:34:36,964::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 03:34:36,964::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110285::WARNING::2014-10-05 03:35:36,964::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 03:35:36,965::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 03:35:36,965::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 03:35:37,820::__init__::232::IOProcess::(_processLogs) (92275) Queuing response >Thread-18::DEBUG::2014-10-05 03:35:37,821::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 03:35:37,821::__init__::232::IOProcess::(_processLogs) (92335) Got request for method 'glob' >Thread-18::DEBUG::2014-10-05 03:35:46,966::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 03:35:46,966::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110286::WARNING::2014-10-05 03:36:46,966::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 03:36:46,966::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 03:36:46,967::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 03:36:56,968::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 03:36:56,968::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110287::WARNING::2014-10-05 03:37:56,968::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 03:37:56,968::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 03:37:56,969::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 03:38:06,970::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 03:38:06,970::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110288::WARNING::2014-10-05 03:39:06,970::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 03:39:06,970::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 03:39:06,971::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 03:39:16,971::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 03:39:16,972::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 03:39:16,972::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110289::WARNING::2014-10-05 03:40:16,972::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 03:40:16,972::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 03:40:16,973::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 03:40:26,974::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 03:40:26,974::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110290::WARNING::2014-10-05 03:41:26,974::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 03:41:26,974::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 03:41:26,975::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 03:41:36,976::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 03:41:36,976::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110291::WARNING::2014-10-05 03:42:36,976::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 03:42:36,976::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 03:42:36,976::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 03:42:46,977::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 03:42:46,978::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110292::WARNING::2014-10-05 03:43:46,977::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 03:43:46,978::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 03:43:46,978::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 03:43:56,979::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 03:43:56,979::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110293::WARNING::2014-10-05 03:44:56,979::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 03:44:56,980::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 03:44:56,980::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 03:45:06,980::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 03:45:06,981::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 03:45:06,981::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-05 03:45:41,980::__init__::232::IOProcess::(_processLogs) (92276) Queuing response >Thread-18::DEBUG::2014-10-05 03:45:41,981::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 03:45:41,981::__init__::232::IOProcess::(_processLogs) (92336) Got request for method 'glob' >Thread-18::DEBUG::2014-10-05 03:45:41,981::__init__::232::IOProcess::(_processLogs) (92277) Queuing response >Thread-18::DEBUG::2014-10-05 03:45:41,981::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 03:45:41,981::__init__::232::IOProcess::(_processLogs) (92337) Got request for method 'glob' >Thread-110294::WARNING::2014-10-05 03:46:06,981::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 03:46:06,982::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 03:46:06,982::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 03:46:16,983::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 03:46:16,983::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110295::WARNING::2014-10-05 03:47:16,983::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 03:47:16,983::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 03:47:16,984::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 03:47:26,985::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 03:47:26,985::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110296::WARNING::2014-10-05 03:48:26,985::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 03:48:26,985::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 03:48:26,986::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 03:48:36,987::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 03:48:36,987::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110297::WARNING::2014-10-05 03:49:36,987::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 03:49:36,987::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 03:49:36,987::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 03:49:46,989::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 03:49:46,989::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110298::WARNING::2014-10-05 03:50:46,989::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 03:50:46,989::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 03:50:46,989::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 03:50:56,990::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 03:50:56,991::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 03:50:56,991::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110299::WARNING::2014-10-05 03:51:56,991::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 03:51:56,991::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 03:51:56,992::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 03:52:06,993::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 03:52:06,993::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110300::WARNING::2014-10-05 03:53:06,993::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 03:53:06,993::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 03:53:06,993::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 03:53:16,995::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 03:53:16,995::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110301::WARNING::2014-10-05 03:54:16,994::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 03:54:16,995::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 03:54:16,995::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 03:54:26,996::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 03:54:26,996::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110302::WARNING::2014-10-05 03:55:26,996::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 03:55:26,997::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 03:55:26,997::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 03:55:36,998::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 03:55:36,998::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-05 03:55:46,140::__init__::232::IOProcess::(_processLogs) (92278) Queuing response >Thread-18::DEBUG::2014-10-05 03:55:46,141::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 03:55:46,141::__init__::232::IOProcess::(_processLogs) (92338) Got request for method 'glob' >Thread-110303::WARNING::2014-10-05 03:56:36,998::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 03:56:36,998::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 03:56:36,999::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 03:56:46,999::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 03:56:47,000::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 03:56:47,000::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110304::WARNING::2014-10-05 03:57:47,000::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 03:57:47,000::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 03:57:47,001::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 03:57:57,002::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 03:57:57,002::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110305::WARNING::2014-10-05 03:58:57,002::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 03:58:57,002::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 03:58:57,002::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 03:59:07,004::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 03:59:07,004::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110306::WARNING::2014-10-05 04:00:07,003::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 04:00:07,004::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 04:00:07,006::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 04:00:17,007::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 04:00:17,007::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110307::WARNING::2014-10-05 04:01:17,007::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 04:01:17,008::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 04:01:17,008::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 04:01:27,009::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 04:01:27,009::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110308::WARNING::2014-10-05 04:02:27,009::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 04:02:27,009::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 04:02:27,010::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 04:02:37,010::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 04:02:37,011::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 04:02:37,011::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110309::WARNING::2014-10-05 04:03:37,011::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 04:03:37,011::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 04:03:37,012::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 04:03:47,013::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 04:03:47,013::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110310::WARNING::2014-10-05 04:04:47,013::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 04:04:47,013::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 04:04:47,013::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 04:04:57,014::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 04:04:57,015::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-05 04:05:52,348::__init__::232::IOProcess::(_processLogs) (92279) Queuing response >Thread-18::DEBUG::2014-10-05 04:05:52,349::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 04:05:52,349::__init__::232::IOProcess::(_processLogs) (92339) Got request for method 'glob' >Thread-18::DEBUG::2014-10-05 04:05:52,349::__init__::232::IOProcess::(_processLogs) (92280) Queuing response >Thread-18::DEBUG::2014-10-05 04:05:52,349::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 04:05:52,349::__init__::232::IOProcess::(_processLogs) (92340) Got request for method 'glob' >Thread-110311::WARNING::2014-10-05 04:05:57,014::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 04:05:57,015::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 04:05:57,015::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 04:06:07,016::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 04:06:07,016::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110312::WARNING::2014-10-05 04:07:07,016::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 04:07:07,016::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 04:07:07,017::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 04:07:17,018::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 04:07:17,018::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110313::WARNING::2014-10-05 04:08:17,018::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 04:08:17,018::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 04:08:17,018::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 04:08:27,019::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 04:08:27,020::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 04:08:27,020::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110314::WARNING::2014-10-05 04:09:27,020::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 04:09:27,020::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 04:09:27,020::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 04:09:37,022::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 04:09:37,022::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110315::WARNING::2014-10-05 04:10:37,021::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 04:10:37,022::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 04:10:37,022::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 04:10:47,023::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 04:10:47,023::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110316::WARNING::2014-10-05 04:11:47,023::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 04:11:47,024::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 04:11:47,024::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 04:11:57,025::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 04:11:57,025::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110317::WARNING::2014-10-05 04:12:57,025::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 04:12:57,026::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 04:12:57,026::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 04:13:07,027::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 04:13:07,027::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110318::WARNING::2014-10-05 04:14:07,027::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 04:14:07,027::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 04:14:07,027::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 04:14:17,028::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 04:14:17,029::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 04:14:17,029::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110319::WARNING::2014-10-05 04:15:17,029::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 04:15:17,029::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 04:15:17,029::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 04:15:27,030::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 04:15:27,031::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-05 04:15:56,508::__init__::232::IOProcess::(_processLogs) (92281) Queuing response >Thread-18::DEBUG::2014-10-05 04:15:56,509::__init__::232::IOProcess::(_processLogs) (92282) Queuing response >Thread-18::DEBUG::2014-10-05 04:15:56,509::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 04:15:56,509::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 04:15:56,509::__init__::232::IOProcess::(_processLogs) (92342) Got request for method 'glob' >Thread-18::DEBUG::2014-10-05 04:15:56,509::__init__::232::IOProcess::(_processLogs) (92341) Got request for method 'glob' >Thread-110320::WARNING::2014-10-05 04:16:27,030::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 04:16:27,031::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 04:16:27,031::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 04:16:37,032::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 04:16:37,033::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110321::WARNING::2014-10-05 04:17:37,032::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 04:17:37,033::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 04:17:37,033::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 04:17:47,034::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 04:17:47,035::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110322::WARNING::2014-10-05 04:18:47,034::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 04:18:47,035::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 04:18:47,035::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 04:18:57,036::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 04:18:57,036::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110323::WARNING::2014-10-05 04:19:57,036::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 04:19:57,037::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 04:19:57,037::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 04:20:07,037::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 04:20:07,038::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 04:20:07,038::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110324::WARNING::2014-10-05 04:21:07,038::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 04:21:07,039::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 04:21:07,039::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 04:21:17,040::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 04:21:17,040::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110325::WARNING::2014-10-05 04:22:17,040::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 04:22:17,041::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 04:22:17,041::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 04:22:27,042::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 04:22:27,043::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110326::WARNING::2014-10-05 04:23:27,042::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 04:23:27,043::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 04:23:27,043::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 04:23:37,044::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 04:23:37,044::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110327::WARNING::2014-10-05 04:24:37,044::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 04:24:37,045::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 04:24:37,045::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 04:24:47,046::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 04:24:47,046::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110328::WARNING::2014-10-05 04:25:47,046::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 04:25:47,047::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 04:25:47,047::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 04:25:57,047::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 04:25:57,048::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 04:25:57,048::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-05 04:26:02,716::__init__::232::IOProcess::(_processLogs) (92283) Queuing response >Thread-18::DEBUG::2014-10-05 04:26:02,717::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 04:26:02,717::__init__::232::IOProcess::(_processLogs) (92343) Got request for method 'glob' >Thread-18::DEBUG::2014-10-05 04:26:02,717::__init__::232::IOProcess::(_processLogs) (92284) Queuing response >Thread-18::DEBUG::2014-10-05 04:26:02,717::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 04:26:02,717::__init__::232::IOProcess::(_processLogs) (92344) Got request for method 'glob' >Thread-110329::WARNING::2014-10-05 04:26:57,048::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 04:26:57,049::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 04:26:57,049::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 04:27:07,050::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 04:27:07,050::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110330::WARNING::2014-10-05 04:28:07,050::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 04:28:07,051::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 04:28:07,051::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 04:28:17,052::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 04:28:17,052::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110331::WARNING::2014-10-05 04:29:17,052::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 04:29:17,052::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 04:29:17,053::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 04:29:27,054::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 04:29:27,054::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110332::WARNING::2014-10-05 04:30:27,054::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 04:30:27,054::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 04:30:27,055::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 04:30:37,056::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 04:30:37,056::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110333::WARNING::2014-10-05 04:31:37,056::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 04:31:37,056::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 04:31:37,056::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 04:31:47,057::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 04:31:47,058::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 04:31:47,058::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110334::WARNING::2014-10-05 04:32:47,058::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 04:32:47,058::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 04:32:47,059::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 04:32:57,060::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 04:32:57,060::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110335::WARNING::2014-10-05 04:33:57,060::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 04:33:57,060::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 04:33:57,060::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 04:34:07,062::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 04:34:07,062::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110336::WARNING::2014-10-05 04:35:07,062::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 04:35:07,062::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 04:35:07,062::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 04:35:17,064::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 04:35:17,064::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-05 04:36:08,924::__init__::232::IOProcess::(_processLogs) (92285) Queuing response >Thread-18::DEBUG::2014-10-05 04:36:08,925::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 04:36:08,925::__init__::232::IOProcess::(_processLogs) (92345) Got request for method 'glob' >Thread-18::DEBUG::2014-10-05 04:36:08,925::__init__::232::IOProcess::(_processLogs) (92286) Queuing response >Thread-18::DEBUG::2014-10-05 04:36:08,925::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 04:36:08,925::__init__::232::IOProcess::(_processLogs) (92346) Got request for method 'glob' >Thread-110337::WARNING::2014-10-05 04:36:17,063::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 04:36:17,064::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 04:36:17,064::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 04:36:27,065::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 04:36:27,066::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110338::WARNING::2014-10-05 04:37:27,065::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 04:37:27,066::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 04:37:27,066::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 04:37:37,066::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 04:37:37,067::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 04:37:37,068::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110339::WARNING::2014-10-05 04:38:37,067::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 04:38:37,068::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 04:38:37,068::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 04:38:47,069::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 04:38:47,070::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110340::WARNING::2014-10-05 04:39:47,069::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 04:39:47,070::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 04:39:47,070::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 04:39:57,071::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 04:39:57,071::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110341::WARNING::2014-10-05 04:40:57,071::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 04:40:57,072::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 04:40:57,072::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 04:41:07,073::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 04:41:07,073::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110342::WARNING::2014-10-05 04:42:07,073::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 04:42:07,073::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 04:42:07,074::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 04:42:17,075::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 04:42:17,075::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110343::WARNING::2014-10-05 04:43:17,075::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 04:43:17,075::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 04:43:17,076::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 04:43:27,076::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 04:43:27,077::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 04:43:27,077::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110344::WARNING::2014-10-05 04:44:27,077::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 04:44:27,077::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 04:44:27,078::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 04:44:37,079::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 04:44:37,079::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110345::WARNING::2014-10-05 04:45:37,079::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 04:45:37,079::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 04:45:37,079::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 04:45:47,081::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 04:45:47,081::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-05 04:46:13,084::__init__::232::IOProcess::(_processLogs) (92287) Queuing response >Thread-18::DEBUG::2014-10-05 04:46:13,085::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 04:46:13,085::__init__::232::IOProcess::(_processLogs) (92347) Got request for method 'glob' >Thread-110346::WARNING::2014-10-05 04:46:47,081::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 04:46:47,081::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 04:46:47,081::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 04:46:57,082::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 04:46:57,083::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110347::WARNING::2014-10-05 04:47:57,082::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 04:47:57,083::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 04:47:57,083::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 04:48:07,084::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 04:48:07,084::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110348::WARNING::2014-10-05 04:49:07,084::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 04:49:07,085::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 04:49:07,085::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 04:49:17,085::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 04:49:17,086::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 04:49:17,086::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110349::WARNING::2014-10-05 04:50:17,086::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 04:50:17,087::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 04:50:17,087::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 04:50:27,088::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 04:50:27,088::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110350::WARNING::2014-10-05 04:51:27,088::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 04:51:27,089::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 04:51:27,089::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 04:51:37,090::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 04:51:37,091::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110351::WARNING::2014-10-05 04:52:37,090::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 04:52:37,091::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 04:52:37,091::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 04:52:47,093::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 04:52:47,093::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110352::WARNING::2014-10-05 04:53:47,092::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 04:53:47,093::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 04:53:47,093::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 04:53:57,094::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 04:53:57,094::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110353::WARNING::2014-10-05 04:54:57,094::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 04:54:57,095::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 04:54:57,095::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 04:55:07,095::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 04:55:07,096::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 04:55:07,096::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110354::WARNING::2014-10-05 04:56:07,096::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 04:56:07,097::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 04:56:07,097::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 04:56:17,098::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 04:56:17,098::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-05 04:56:17,244::__init__::232::IOProcess::(_processLogs) (92288) Queuing response >Thread-18::DEBUG::2014-10-05 04:56:17,244::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 04:56:17,245::__init__::232::IOProcess::(_processLogs) (92348) Got request for method 'glob' >Thread-18::DEBUG::2014-10-05 04:56:17,245::__init__::232::IOProcess::(_processLogs) (92289) Queuing response >Thread-18::DEBUG::2014-10-05 04:56:17,245::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 04:56:17,245::__init__::232::IOProcess::(_processLogs) (92349) Got request for method 'glob' >Thread-110355::WARNING::2014-10-05 04:57:17,098::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 04:57:17,098::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 04:57:17,098::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 04:57:27,100::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 04:57:27,100::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110356::WARNING::2014-10-05 04:58:27,100::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 04:58:27,100::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 04:58:27,100::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 04:58:37,101::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 04:58:37,102::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110357::WARNING::2014-10-05 04:59:37,101::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 04:59:37,102::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 04:59:37,102::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 04:59:47,103::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 04:59:47,103::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110358::WARNING::2014-10-05 05:00:47,103::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 05:00:47,104::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 05:00:47,104::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 05:00:57,104::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 05:00:57,105::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 05:00:57,105::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110359::WARNING::2014-10-05 05:01:57,105::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 05:01:57,106::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 05:01:57,106::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 05:02:07,107::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 05:02:07,107::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110360::WARNING::2014-10-05 05:03:07,107::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 05:03:07,108::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 05:03:07,108::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 05:03:17,109::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 05:03:17,109::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110361::WARNING::2014-10-05 05:04:17,109::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 05:04:17,109::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 05:04:17,110::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 05:04:27,111::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 05:04:27,111::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110362::WARNING::2014-10-05 05:05:27,111::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 05:05:27,111::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 05:05:27,111::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 05:05:37,112::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 05:05:37,113::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-05 05:06:23,452::__init__::232::IOProcess::(_processLogs) (92290) Queuing response >Thread-18::DEBUG::2014-10-05 05:06:23,453::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 05:06:23,453::__init__::232::IOProcess::(_processLogs) (92350) Got request for method 'glob' >Thread-18::DEBUG::2014-10-05 05:06:23,453::__init__::232::IOProcess::(_processLogs) (92291) Queuing response >Thread-18::DEBUG::2014-10-05 05:06:23,453::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 05:06:23,453::__init__::232::IOProcess::(_processLogs) (92351) Got request for method 'glob' >Thread-110363::WARNING::2014-10-05 05:06:37,112::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 05:06:37,113::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 05:06:37,113::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 05:06:47,113::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 05:06:47,114::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 05:06:47,115::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110364::WARNING::2014-10-05 05:07:47,114::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 05:07:47,115::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 05:07:47,115::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 05:07:57,116::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 05:07:57,116::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110365::WARNING::2014-10-05 05:08:57,116::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 05:08:57,116::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 05:08:57,117::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 05:09:07,118::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 05:09:07,118::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110366::WARNING::2014-10-05 05:10:07,118::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 05:10:07,118::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 05:10:07,120::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 05:10:17,121::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 05:10:17,121::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110367::WARNING::2014-10-05 05:11:17,121::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 05:11:17,121::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 05:11:17,122::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 05:11:27,123::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 05:11:27,123::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110368::WARNING::2014-10-05 05:12:27,123::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 05:12:27,123::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 05:12:27,124::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 05:12:37,124::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 05:12:37,125::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 05:12:37,125::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110369::WARNING::2014-10-05 05:13:37,125::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 05:13:37,125::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 05:13:37,126::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 05:13:47,127::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 05:13:47,127::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110370::WARNING::2014-10-05 05:14:47,127::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 05:14:47,127::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 05:14:47,127::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 05:14:57,129::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 05:14:57,129::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110371::WARNING::2014-10-05 05:15:57,129::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 05:15:57,129::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 05:15:57,129::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 05:16:07,131::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 05:16:07,131::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-05 05:16:27,612::__init__::232::IOProcess::(_processLogs) (92292) Queuing response >Thread-18::DEBUG::2014-10-05 05:16:27,613::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 05:16:27,613::__init__::232::IOProcess::(_processLogs) (92352) Got request for method 'glob' >Thread-18::DEBUG::2014-10-05 05:16:27,613::__init__::232::IOProcess::(_processLogs) (92293) Queuing response >Thread-18::DEBUG::2014-10-05 05:16:27,613::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 05:16:27,613::__init__::232::IOProcess::(_processLogs) (92353) Got request for method 'glob' >Thread-18::DEBUG::2014-10-05 05:16:27,613::__init__::232::IOProcess::(_processLogs) (92294) Queuing response >Thread-18::DEBUG::2014-10-05 05:16:27,613::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 05:16:27,613::__init__::232::IOProcess::(_processLogs) (92354) Got request for method 'glob' >Thread-110372::WARNING::2014-10-05 05:17:07,131::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 05:17:07,131::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 05:17:07,131::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 05:17:17,132::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 05:17:17,133::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110373::WARNING::2014-10-05 05:18:17,132::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 05:18:17,133::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 05:18:17,133::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 05:18:27,134::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 05:18:27,135::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 05:18:27,135::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110374::WARNING::2014-10-05 05:19:27,135::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 05:19:27,135::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 05:19:27,135::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 05:19:37,136::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 05:19:37,137::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110375::WARNING::2014-10-05 05:20:37,136::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 05:20:37,137::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 05:20:37,137::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 05:20:47,138::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 05:20:47,139::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110376::WARNING::2014-10-05 05:21:47,138::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 05:21:47,139::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 05:21:47,139::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 05:21:57,140::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 05:21:57,141::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110377::WARNING::2014-10-05 05:22:57,140::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 05:22:57,141::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 05:22:57,141::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 05:23:07,142::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 05:23:07,143::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110378::WARNING::2014-10-05 05:24:07,142::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 05:24:07,143::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 05:24:07,143::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 05:24:17,143::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 05:24:17,144::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 05:24:17,145::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110379::WARNING::2014-10-05 05:25:17,144::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 05:25:17,145::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 05:25:17,145::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 05:25:27,146::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 05:25:27,146::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110380::WARNING::2014-10-05 05:26:27,146::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 05:26:27,147::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 05:26:27,147::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 05:26:31,772::__init__::232::IOProcess::(_processLogs) (92295) Queuing response >Thread-18::DEBUG::2014-10-05 05:26:31,773::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 05:26:31,773::__init__::232::IOProcess::(_processLogs) (92355) Got request for method 'glob' >Thread-18::DEBUG::2014-10-05 05:26:37,148::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 05:26:37,148::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110381::WARNING::2014-10-05 05:27:37,148::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 05:27:37,149::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 05:27:37,149::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 05:27:47,150::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 05:27:47,150::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110382::WARNING::2014-10-05 05:28:47,150::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 05:28:47,151::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 05:28:47,151::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 05:28:57,152::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 05:28:57,152::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110383::WARNING::2014-10-05 05:29:57,152::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 05:29:57,153::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 05:29:57,153::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 05:30:07,153::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 05:30:07,154::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 05:30:07,154::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110384::WARNING::2014-10-05 05:31:07,154::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 05:31:07,155::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 05:31:07,155::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 05:31:17,156::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 05:31:17,156::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110385::WARNING::2014-10-05 05:32:17,156::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 05:32:17,157::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 05:32:17,157::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 05:32:27,158::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 05:32:27,158::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110386::WARNING::2014-10-05 05:33:27,158::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 05:33:27,158::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 05:33:27,159::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 05:33:37,160::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 05:33:37,160::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110387::WARNING::2014-10-05 05:34:37,160::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 05:34:37,160::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 05:34:37,161::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 05:34:47,162::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 05:34:47,162::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110388::WARNING::2014-10-05 05:35:47,162::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 05:35:47,162::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 05:35:47,162::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 05:35:57,163::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 05:35:57,164::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 05:35:57,164::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-05 05:36:35,932::__init__::232::IOProcess::(_processLogs) (92296) Queuing response >Thread-18::DEBUG::2014-10-05 05:36:35,933::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 05:36:35,933::__init__::232::IOProcess::(_processLogs) (92356) Got request for method 'glob' >Thread-110389::WARNING::2014-10-05 05:36:57,164::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 05:36:57,164::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 05:36:57,164::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 05:37:07,166::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 05:37:07,166::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110390::WARNING::2014-10-05 05:38:07,166::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 05:38:07,166::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 05:38:07,166::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 05:38:17,167::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 05:38:17,168::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110391::WARNING::2014-10-05 05:39:17,167::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 05:39:17,168::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 05:39:17,168::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 05:39:27,169::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 05:39:27,170::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110392::WARNING::2014-10-05 05:40:27,169::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 05:40:27,170::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 05:40:27,170::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 05:40:37,171::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 05:40:37,171::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110393::WARNING::2014-10-05 05:41:37,171::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 05:41:37,172::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 05:41:37,172::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 05:41:47,172::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 05:41:47,173::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 05:41:47,173::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110394::WARNING::2014-10-05 05:42:47,173::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 05:42:47,174::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 05:42:47,174::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 05:42:57,175::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 05:42:57,175::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110395::WARNING::2014-10-05 05:43:57,175::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 05:43:57,176::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 05:43:57,176::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 05:44:07,177::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 05:44:07,177::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110396::WARNING::2014-10-05 05:45:07,177::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 05:45:07,177::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 05:45:07,178::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 05:45:17,179::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 05:45:17,179::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110397::WARNING::2014-10-05 05:46:17,179::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 05:46:17,179::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 05:46:17,179::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 05:46:27,181::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 05:46:27,181::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-05 05:46:40,093::__init__::232::IOProcess::(_processLogs) (92297) Queuing response >Thread-18::DEBUG::2014-10-05 05:46:40,093::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 05:46:40,093::__init__::232::IOProcess::(_processLogs) (92357) Got request for method 'glob' >Thread-110398::WARNING::2014-10-05 05:47:27,180::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 05:47:27,181::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 05:47:27,181::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 05:47:37,182::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 05:47:37,183::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 05:47:37,183::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110399::WARNING::2014-10-05 05:48:37,182::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 05:48:37,183::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 05:48:37,183::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 05:48:47,184::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 05:48:47,185::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110400::WARNING::2014-10-05 05:49:47,184::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 05:49:47,185::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 05:49:47,185::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 05:49:57,186::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 05:49:57,187::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110401::WARNING::2014-10-05 05:50:57,186::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 05:50:57,187::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 05:50:57,187::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 05:51:07,188::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 05:51:07,188::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110402::WARNING::2014-10-05 05:52:07,188::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 05:52:07,189::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 05:52:07,189::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 05:52:17,190::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 05:52:17,191::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110403::WARNING::2014-10-05 05:53:17,190::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 05:53:17,191::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 05:53:17,191::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 05:53:27,191::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 05:53:27,192::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 05:53:27,192::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110404::WARNING::2014-10-05 05:54:27,192::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 05:54:27,192::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 05:54:27,193::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 05:54:37,194::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 05:54:37,194::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110405::WARNING::2014-10-05 05:55:37,194::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 05:55:37,194::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 05:55:37,194::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 05:55:47,196::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 05:55:47,196::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-05 05:56:44,252::__init__::232::IOProcess::(_processLogs) (92298) Queuing response >Thread-18::DEBUG::2014-10-05 05:56:44,253::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 05:56:44,253::__init__::232::IOProcess::(_processLogs) (92358) Got request for method 'glob' >Thread-18::DEBUG::2014-10-05 05:56:44,253::__init__::232::IOProcess::(_processLogs) (92299) Queuing response >Thread-18::DEBUG::2014-10-05 05:56:44,253::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 05:56:44,253::__init__::232::IOProcess::(_processLogs) (92359) Got request for method 'glob' >Thread-110406::WARNING::2014-10-05 05:56:47,195::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 05:56:47,196::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 05:56:47,196::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 05:56:57,197::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 05:56:57,197::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110407::WARNING::2014-10-05 05:57:57,197::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 05:57:57,198::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 05:57:57,198::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 05:58:07,199::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 05:58:07,199::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110408::WARNING::2014-10-05 05:59:07,199::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 05:59:07,199::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 05:59:07,200::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 05:59:17,200::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 05:59:17,201::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 05:59:17,201::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110409::WARNING::2014-10-05 06:00:17,201::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 06:00:17,201::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 06:00:17,202::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 06:00:27,203::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 06:00:27,203::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110410::WARNING::2014-10-05 06:01:27,203::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 06:01:27,203::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 06:01:27,203::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 06:01:37,205::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 06:01:37,205::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110411::WARNING::2014-10-05 06:02:37,205::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 06:02:37,205::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 06:02:37,205::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 06:02:47,206::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 06:02:47,207::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110412::WARNING::2014-10-05 06:03:47,206::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 06:03:47,207::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 06:03:47,207::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 06:03:57,208::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 06:03:57,208::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110413::WARNING::2014-10-05 06:04:57,208::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 06:04:57,209::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 06:04:57,209::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 06:05:07,209::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 06:05:07,210::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 06:05:07,210::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110414::WARNING::2014-10-05 06:06:07,210::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 06:06:07,211::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 06:06:07,211::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 06:06:17,212::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 06:06:17,212::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-05 06:06:50,461::__init__::232::IOProcess::(_processLogs) (92301) Queuing response >Thread-18::DEBUG::2014-10-05 06:06:50,461::__init__::232::IOProcess::(_processLogs) (92300) Queuing response >Thread-18::DEBUG::2014-10-05 06:06:50,462::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 06:06:50,462::__init__::232::IOProcess::(_processLogs) (92360) Got request for method 'glob' >Thread-18::DEBUG::2014-10-05 06:06:50,463::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 06:06:50,463::__init__::232::IOProcess::(_processLogs) (92361) Got request for method 'glob' >Thread-110415::WARNING::2014-10-05 06:07:17,212::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 06:07:17,212::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 06:07:17,213::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 06:07:27,214::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 06:07:27,214::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110416::WARNING::2014-10-05 06:08:27,214::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 06:08:27,215::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 06:08:27,215::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 06:08:37,216::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 06:08:37,216::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110417::WARNING::2014-10-05 06:09:37,216::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 06:09:37,216::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 06:09:37,217::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 06:09:47,218::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 06:09:47,218::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110418::WARNING::2014-10-05 06:10:47,218::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 06:10:47,218::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 06:10:47,218::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 06:10:57,219::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 06:10:57,220::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 06:10:57,220::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110419::WARNING::2014-10-05 06:11:57,220::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 06:11:57,220::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 06:11:57,220::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 06:12:07,222::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 06:12:07,222::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110420::WARNING::2014-10-05 06:13:07,222::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 06:13:07,222::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 06:13:07,222::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 06:13:17,223::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 06:13:17,224::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110421::WARNING::2014-10-05 06:14:17,223::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 06:14:17,224::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 06:14:17,224::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 06:14:27,226::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 06:14:27,226::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110422::WARNING::2014-10-05 06:15:27,226::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 06:15:27,226::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 06:15:27,227::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 06:15:37,228::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 06:15:37,228::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110423::WARNING::2014-10-05 06:16:37,228::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 06:16:37,228::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 06:16:37,229::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 06:16:47,229::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 06:16:47,230::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 06:16:47,230::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-05 06:16:56,668::__init__::232::IOProcess::(_processLogs) (92302) Queuing response >Thread-18::DEBUG::2014-10-05 06:16:56,669::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 06:16:56,669::__init__::232::IOProcess::(_processLogs) (92362) Got request for method 'glob' >Thread-18::DEBUG::2014-10-05 06:16:56,669::__init__::232::IOProcess::(_processLogs) (92303) Queuing response >Thread-18::DEBUG::2014-10-05 06:16:56,669::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 06:16:56,669::__init__::232::IOProcess::(_processLogs) (92363) Got request for method 'glob' >Thread-110424::WARNING::2014-10-05 06:17:47,230::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 06:17:47,230::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 06:17:47,231::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 06:17:57,232::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 06:17:57,232::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110425::WARNING::2014-10-05 06:18:57,232::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 06:18:57,233::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 06:18:57,233::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 06:19:07,234::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 06:19:07,234::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110426::WARNING::2014-10-05 06:20:07,234::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 06:20:07,234::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 06:20:07,235::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 06:20:17,236::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 06:20:17,236::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110427::WARNING::2014-10-05 06:21:17,236::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 06:21:17,236::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 06:21:17,237::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 06:21:27,238::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 06:21:27,238::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110428::WARNING::2014-10-05 06:22:27,238::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 06:22:27,239::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 06:22:27,239::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 06:22:37,239::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 06:22:37,240::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 06:22:37,240::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110429::WARNING::2014-10-05 06:23:37,240::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 06:23:37,241::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 06:23:37,241::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 06:23:47,242::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 06:23:47,242::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110430::WARNING::2014-10-05 06:24:47,242::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 06:24:47,242::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 06:24:47,243::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 06:24:57,244::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 06:24:57,244::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110431::WARNING::2014-10-05 06:25:57,244::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 06:25:57,244::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 06:25:57,245::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 06:26:07,246::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 06:26:07,246::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-05 06:27:00,829::__init__::232::IOProcess::(_processLogs) (92304) Queuing response >Thread-18::DEBUG::2014-10-05 06:27:00,829::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 06:27:00,829::__init__::232::IOProcess::(_processLogs) (92364) Got request for method 'glob' >Thread-18::DEBUG::2014-10-05 06:27:00,829::__init__::232::IOProcess::(_processLogs) (92305) Queuing response >Thread-18::DEBUG::2014-10-05 06:27:00,829::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 06:27:00,829::__init__::232::IOProcess::(_processLogs) (92365) Got request for method 'glob' >Thread-110432::WARNING::2014-10-05 06:27:07,246::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 06:27:07,246::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 06:27:07,247::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 06:27:17,248::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 06:27:17,249::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110433::WARNING::2014-10-05 06:28:17,248::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 06:28:17,249::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 06:28:17,249::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 06:28:27,249::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 06:28:27,250::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 06:28:27,251::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110434::WARNING::2014-10-05 06:29:27,250::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 06:29:27,251::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 06:29:27,251::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 06:29:37,252::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 06:29:37,252::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110435::WARNING::2014-10-05 06:30:37,252::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 06:30:37,253::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 06:30:37,253::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 06:30:47,254::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 06:30:47,254::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110436::WARNING::2014-10-05 06:31:47,254::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 06:31:47,254::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 06:31:47,255::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 06:31:57,256::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 06:31:57,256::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110437::WARNING::2014-10-05 06:32:57,256::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 06:32:57,256::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 06:32:57,257::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 06:33:07,258::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 06:33:07,258::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110438::WARNING::2014-10-05 06:34:07,258::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 06:34:07,258::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 06:34:07,258::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 06:34:17,259::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 06:34:17,260::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 06:34:17,260::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110439::WARNING::2014-10-05 06:35:17,260::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 06:35:17,260::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 06:35:17,260::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 06:35:27,262::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 06:35:27,262::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110440::WARNING::2014-10-05 06:36:27,262::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 06:36:27,262::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 06:36:27,262::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 06:36:37,263::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 06:36:37,264::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-05 06:37:04,988::__init__::232::IOProcess::(_processLogs) (92306) Queuing response >Thread-18::DEBUG::2014-10-05 06:37:04,989::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 06:37:04,989::__init__::232::IOProcess::(_processLogs) (92366) Got request for method 'glob' >Thread-110441::WARNING::2014-10-05 06:37:37,263::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 06:37:37,264::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 06:37:37,264::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 06:37:47,265::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 06:37:47,265::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110442::WARNING::2014-10-05 06:38:47,265::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 06:38:47,266::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 06:38:47,266::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 06:38:57,267::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 06:38:57,267::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110443::WARNING::2014-10-05 06:39:57,267::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 06:39:57,268::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 06:39:57,268::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 06:40:07,269::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 06:40:07,270::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 06:40:07,270::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110444::WARNING::2014-10-05 06:41:07,270::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 06:41:07,270::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 06:41:07,270::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 06:41:17,271::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 06:41:17,272::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110445::WARNING::2014-10-05 06:42:17,271::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 06:42:17,272::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 06:42:17,272::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 06:42:27,273::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 06:42:27,274::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110446::WARNING::2014-10-05 06:43:27,273::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 06:43:27,274::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 06:43:27,274::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 06:43:37,275::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 06:43:37,276::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110447::WARNING::2014-10-05 06:44:37,275::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 06:44:37,276::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 06:44:37,276::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 06:44:47,278::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 06:44:47,278::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110448::WARNING::2014-10-05 06:45:47,277::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 06:45:47,278::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 06:45:47,278::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 06:45:57,279::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 06:45:57,280::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 06:45:57,280::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110449::WARNING::2014-10-05 06:46:57,280::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 06:46:57,280::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 06:46:57,280::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 06:47:07,282::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 06:47:07,282::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-05 06:47:09,148::__init__::232::IOProcess::(_processLogs) (92307) Queuing response >Thread-18::DEBUG::2014-10-05 06:47:09,148::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 06:47:09,149::__init__::232::IOProcess::(_processLogs) (92367) Got request for method 'glob' >Thread-110450::WARNING::2014-10-05 06:48:07,282::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 06:48:07,282::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 06:48:07,282::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 06:48:17,284::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 06:48:17,284::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110451::WARNING::2014-10-05 06:49:17,284::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 06:49:17,284::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 06:49:17,284::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 06:49:27,286::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 06:49:27,286::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110452::WARNING::2014-10-05 06:50:27,286::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 06:50:27,286::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 06:50:27,286::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 06:50:37,288::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 06:50:37,288::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110453::WARNING::2014-10-05 06:51:37,288::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 06:51:37,288::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 06:51:37,289::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 06:51:47,289::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 06:51:47,290::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 06:51:47,290::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110454::WARNING::2014-10-05 06:52:47,290::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 06:52:47,290::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 06:52:47,291::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 06:52:57,292::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 06:52:57,292::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110455::WARNING::2014-10-05 06:53:57,291::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 06:53:57,292::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 06:53:57,292::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 06:54:07,293::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 06:54:07,293::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110456::WARNING::2014-10-05 06:55:07,293::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 06:55:07,294::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 06:55:07,294::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 06:55:17,295::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 06:55:17,295::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110457::WARNING::2014-10-05 06:56:17,295::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 06:56:17,295::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 06:56:17,296::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 06:56:27,297::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 06:56:27,297::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-05 06:57:15,356::__init__::232::IOProcess::(_processLogs) (92308) Queuing response >Thread-18::DEBUG::2014-10-05 06:57:15,357::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 06:57:15,357::__init__::232::IOProcess::(_processLogs) (92368) Got request for method 'glob' >Thread-18::DEBUG::2014-10-05 06:57:15,357::__init__::232::IOProcess::(_processLogs) (92309) Queuing response >Thread-18::DEBUG::2014-10-05 06:57:15,357::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 06:57:15,357::__init__::232::IOProcess::(_processLogs) (92369) Got request for method 'glob' >Thread-110458::WARNING::2014-10-05 06:57:27,297::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 06:57:27,297::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 06:57:27,297::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 06:57:37,298::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 06:57:37,299::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 06:57:37,299::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110459::WARNING::2014-10-05 06:58:37,299::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 06:58:37,299::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 06:58:37,299::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 06:58:47,300::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 06:58:47,301::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110460::WARNING::2014-10-05 06:59:47,300::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 06:59:47,301::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 06:59:47,301::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 06:59:57,302::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 06:59:57,302::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110461::WARNING::2014-10-05 07:00:57,302::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 07:00:57,302::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 07:00:57,303::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 07:01:07,304::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 07:01:07,304::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110462::WARNING::2014-10-05 07:02:07,304::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 07:02:07,304::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 07:02:07,304::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 07:02:17,306::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 07:02:17,306::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110463::WARNING::2014-10-05 07:03:17,306::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 07:03:17,306::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 07:03:17,306::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 07:03:27,307::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 07:03:27,308::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 07:03:27,308::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110464::WARNING::2014-10-05 07:04:27,308::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 07:04:27,308::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 07:04:27,308::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 07:04:37,310::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 07:04:37,310::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110465::WARNING::2014-10-05 07:05:37,310::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 07:05:37,310::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 07:05:37,310::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 07:05:47,311::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 07:05:47,312::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110466::WARNING::2014-10-05 07:06:47,311::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 07:06:47,312::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 07:06:47,312::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 07:06:57,313::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 07:06:57,313::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-05 07:07:19,516::__init__::232::IOProcess::(_processLogs) (92311) Queuing response >Thread-18::DEBUG::2014-10-05 07:07:19,517::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 07:07:19,517::__init__::232::IOProcess::(_processLogs) (92370) Got request for method 'glob' >Thread-18::DEBUG::2014-10-05 07:07:19,517::__init__::232::IOProcess::(_processLogs) (92310) Queuing response >Thread-18::DEBUG::2014-10-05 07:07:19,517::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 07:07:19,517::__init__::232::IOProcess::(_processLogs) (92371) Got request for method 'glob' >Thread-110467::WARNING::2014-10-05 07:07:57,313::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 07:07:57,314::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 07:07:57,314::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 07:08:07,315::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 07:08:07,315::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110468::WARNING::2014-10-05 07:09:07,315::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 07:09:07,316::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 07:09:07,316::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 07:09:17,316::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 07:09:17,317::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 07:09:17,317::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110469::WARNING::2014-10-05 07:10:17,317::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 07:10:17,318::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 07:10:17,318::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 07:10:27,319::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 07:10:27,319::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110470::WARNING::2014-10-05 07:11:27,319::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 07:11:27,319::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 07:11:27,320::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 07:11:37,321::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 07:11:37,321::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110471::WARNING::2014-10-05 07:12:37,321::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 07:12:37,322::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 07:12:37,322::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 07:12:47,323::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 07:12:47,323::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110472::WARNING::2014-10-05 07:13:47,323::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 07:13:47,324::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 07:13:47,324::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 07:13:57,325::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 07:13:57,325::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110473::WARNING::2014-10-05 07:14:57,325::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 07:14:57,326::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 07:14:57,326::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 07:15:07,326::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 07:15:07,327::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 07:15:07,327::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110474::WARNING::2014-10-05 07:16:07,327::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 07:16:07,328::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 07:16:07,328::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 07:16:17,329::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 07:16:17,330::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110475::WARNING::2014-10-05 07:17:17,329::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 07:17:17,330::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 07:17:17,330::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 07:17:23,676::__init__::232::IOProcess::(_processLogs) (92312) Queuing response >Thread-18::DEBUG::2014-10-05 07:17:23,677::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 07:17:23,677::__init__::232::IOProcess::(_processLogs) (92372) Got request for method 'glob' >Thread-18::DEBUG::2014-10-05 07:17:23,677::__init__::232::IOProcess::(_processLogs) (92313) Queuing response >Thread-18::DEBUG::2014-10-05 07:17:23,677::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 07:17:23,677::__init__::232::IOProcess::(_processLogs) (92373) Got request for method 'glob' >Thread-18::DEBUG::2014-10-05 07:17:27,331::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 07:17:27,332::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110476::WARNING::2014-10-05 07:18:27,331::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 07:18:27,332::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 07:18:27,332::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 07:18:37,333::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 07:18:37,334::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110477::WARNING::2014-10-05 07:19:37,333::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 07:19:37,334::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 07:19:37,334::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 07:19:47,335::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 07:19:47,336::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110478::WARNING::2014-10-05 07:20:47,335::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 07:20:47,336::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 07:20:47,336::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 07:20:57,336::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 07:20:57,337::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 07:20:57,338::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110479::WARNING::2014-10-05 07:21:57,337::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 07:21:57,338::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 07:21:57,338::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 07:22:07,340::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 07:22:07,340::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110480::WARNING::2014-10-05 07:23:07,340::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 07:23:07,340::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 07:23:07,340::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 07:23:17,341::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 07:23:17,342::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110481::WARNING::2014-10-05 07:24:17,341::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 07:24:17,342::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 07:24:17,342::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 07:24:27,343::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 07:24:27,344::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110482::WARNING::2014-10-05 07:25:27,343::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 07:25:27,344::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 07:25:27,344::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 07:25:37,345::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 07:25:37,345::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110483::WARNING::2014-10-05 07:26:37,345::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 07:26:37,346::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 07:26:37,346::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 07:26:47,346::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 07:26:47,347::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 07:26:47,347::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-05 07:27:27,836::__init__::232::IOProcess::(_processLogs) (92314) Queuing response >Thread-18::DEBUG::2014-10-05 07:27:27,837::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 07:27:27,837::__init__::232::IOProcess::(_processLogs) (92374) Got request for method 'glob' >Thread-18::DEBUG::2014-10-05 07:27:27,837::__init__::232::IOProcess::(_processLogs) (92315) Queuing response >Thread-18::DEBUG::2014-10-05 07:27:27,837::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 07:27:27,837::__init__::232::IOProcess::(_processLogs) (92375) Got request for method 'glob' >Thread-110484::WARNING::2014-10-05 07:27:47,347::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 07:27:47,348::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 07:27:47,348::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 07:27:57,349::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 07:27:57,349::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110485::WARNING::2014-10-05 07:28:57,349::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 07:28:57,350::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 07:28:57,350::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 07:29:07,351::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 07:29:07,351::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110486::WARNING::2014-10-05 07:30:07,351::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 07:30:07,352::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 07:30:07,352::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 07:30:17,353::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 07:30:17,353::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110487::WARNING::2014-10-05 07:31:17,353::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 07:31:17,353::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 07:31:17,354::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 07:31:27,355::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 07:31:27,355::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110488::WARNING::2014-10-05 07:32:27,355::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 07:32:27,355::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 07:32:27,355::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 07:32:37,356::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 07:32:37,357::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 07:32:37,357::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110489::WARNING::2014-10-05 07:33:37,357::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 07:33:37,357::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 07:33:37,357::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 07:33:47,359::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 07:33:47,359::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110490::WARNING::2014-10-05 07:34:47,359::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 07:34:47,359::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 07:34:47,359::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 07:34:57,360::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 07:34:57,361::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110491::WARNING::2014-10-05 07:35:57,360::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 07:35:57,361::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 07:35:57,361::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 07:36:07,362::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 07:36:07,362::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110492::WARNING::2014-10-05 07:37:07,362::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 07:37:07,363::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 07:37:07,363::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 07:37:17,364::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 07:37:17,364::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-05 07:37:34,044::__init__::232::IOProcess::(_processLogs) (92316) Queuing response >Thread-18::DEBUG::2014-10-05 07:37:34,045::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 07:37:34,045::__init__::232::IOProcess::(_processLogs) (92376) Got request for method 'glob' >Thread-18::DEBUG::2014-10-05 07:37:34,045::__init__::232::IOProcess::(_processLogs) (92317) Queuing response >Thread-18::DEBUG::2014-10-05 07:37:34,045::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 07:37:34,045::__init__::232::IOProcess::(_processLogs) (92377) Got request for method 'glob' >Thread-110493::WARNING::2014-10-05 07:38:17,364::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 07:38:17,365::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 07:38:17,365::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 07:38:27,365::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 07:38:27,366::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 07:38:27,366::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110494::WARNING::2014-10-05 07:39:27,366::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 07:39:27,367::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 07:39:27,367::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 07:39:37,368::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 07:39:37,368::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110495::WARNING::2014-10-05 07:40:37,368::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 07:40:37,368::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 07:40:37,369::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 07:40:47,370::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 07:40:47,370::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110496::WARNING::2014-10-05 07:41:47,370::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 07:41:47,370::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 07:41:47,371::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 07:41:57,372::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 07:41:57,372::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110497::WARNING::2014-10-05 07:42:57,372::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 07:42:57,372::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 07:42:57,372::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 07:43:07,374::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 07:43:07,374::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110498::WARNING::2014-10-05 07:44:07,374::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 07:44:07,374::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 07:44:07,374::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 07:44:17,375::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 07:44:17,376::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 07:44:17,376::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110499::WARNING::2014-10-05 07:45:17,376::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 07:45:17,376::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 07:45:17,377::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 07:45:27,378::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 07:45:27,378::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110500::WARNING::2014-10-05 07:46:27,378::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 07:46:27,378::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 07:46:27,379::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 07:46:37,380::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 07:46:37,380::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110501::WARNING::2014-10-05 07:47:37,380::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 07:47:37,380::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 07:47:37,380::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 07:47:38,204::__init__::232::IOProcess::(_processLogs) (92318) Queuing response >Thread-18::DEBUG::2014-10-05 07:47:38,205::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 07:47:38,205::__init__::232::IOProcess::(_processLogs) (92378) Got request for method 'glob' >Thread-18::DEBUG::2014-10-05 07:47:38,205::__init__::232::IOProcess::(_processLogs) (92319) Queuing response >Thread-18::DEBUG::2014-10-05 07:47:38,205::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 07:47:38,205::__init__::232::IOProcess::(_processLogs) (92379) Got request for method 'glob' >Thread-18::DEBUG::2014-10-05 07:47:47,382::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 07:47:47,382::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110502::WARNING::2014-10-05 07:48:47,382::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 07:48:47,382::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 07:48:47,382::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 07:48:57,384::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 07:48:57,384::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110503::WARNING::2014-10-05 07:49:57,384::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 07:49:57,384::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 07:49:57,384::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 07:50:07,385::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 07:50:07,386::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 07:50:07,386::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110504::WARNING::2014-10-05 07:51:07,386::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 07:51:07,386::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 07:51:07,386::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 07:51:17,388::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 07:51:17,388::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110505::WARNING::2014-10-05 07:52:17,388::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 07:52:17,388::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 07:52:17,388::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 07:52:27,390::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 07:52:27,390::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110506::WARNING::2014-10-05 07:53:27,390::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 07:53:27,390::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 07:53:27,390::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 07:53:37,391::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 07:53:37,392::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110507::WARNING::2014-10-05 07:54:37,391::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 07:54:37,392::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 07:54:37,392::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 07:54:47,393::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 07:54:47,393::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110508::WARNING::2014-10-05 07:55:47,393::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 07:55:47,394::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 07:55:47,394::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 07:55:57,394::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 07:55:57,395::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 07:55:57,395::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110509::WARNING::2014-10-05 07:56:57,395::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 07:56:57,396::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 07:56:57,396::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 07:57:07,397::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 07:57:07,397::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-05 07:57:42,364::__init__::232::IOProcess::(_processLogs) (92321) Queuing response >Thread-18::DEBUG::2014-10-05 07:57:42,365::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 07:57:42,365::__init__::232::IOProcess::(_processLogs) (92380) Got request for method 'glob' >Thread-18::DEBUG::2014-10-05 07:57:42,365::__init__::232::IOProcess::(_processLogs) (92320) Queuing response >Thread-18::DEBUG::2014-10-05 07:57:42,365::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 07:57:42,365::__init__::232::IOProcess::(_processLogs) (92381) Got request for method 'glob' >Thread-110510::WARNING::2014-10-05 07:58:07,397::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 07:58:07,398::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 07:58:07,398::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 07:58:17,400::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 07:58:17,401::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110511::WARNING::2014-10-05 07:59:17,400::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 07:59:17,400::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 07:59:17,400::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 07:59:27,402::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 07:59:27,402::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110512::WARNING::2014-10-05 08:00:27,401::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 08:00:27,402::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 08:00:27,402::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 08:00:37,403::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 08:00:37,404::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110513::WARNING::2014-10-05 08:01:37,403::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 08:01:37,404::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 08:01:37,404::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 08:01:47,405::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 08:01:47,405::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 08:01:47,406::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110514::WARNING::2014-10-05 08:02:47,405::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 08:02:47,406::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 08:02:47,406::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 08:02:57,407::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 08:02:57,408::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110515::WARNING::2014-10-05 08:03:57,407::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 08:03:57,408::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 08:03:57,408::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 08:04:07,409::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 08:04:07,409::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110516::WARNING::2014-10-05 08:05:07,409::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 08:05:07,410::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 08:05:07,410::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 08:05:17,411::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 08:05:17,411::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110517::WARNING::2014-10-05 08:06:17,411::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 08:06:17,412::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 08:06:17,412::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 08:06:27,413::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 08:06:27,413::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110518::WARNING::2014-10-05 08:07:27,413::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 08:07:27,413::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 08:07:27,414::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 08:07:37,414::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 08:07:37,415::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 08:07:37,415::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-05 08:07:46,524::__init__::232::IOProcess::(_processLogs) (92322) Queuing response >Thread-18::DEBUG::2014-10-05 08:07:46,525::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 08:07:46,525::__init__::232::IOProcess::(_processLogs) (92382) Got request for method 'glob' >Thread-110519::WARNING::2014-10-05 08:08:37,415::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 08:08:37,415::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 08:08:37,416::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 08:08:47,417::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 08:08:47,417::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110520::WARNING::2014-10-05 08:09:47,417::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 08:09:47,417::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 08:09:47,418::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 08:09:57,419::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 08:09:57,419::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110521::WARNING::2014-10-05 08:10:57,419::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 08:10:57,419::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 08:10:57,420::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 08:11:07,421::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 08:11:07,421::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110522::WARNING::2014-10-05 08:12:07,421::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 08:12:07,421::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 08:12:07,421::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 08:12:17,423::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 08:12:17,423::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110523::WARNING::2014-10-05 08:13:17,422::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 08:13:17,423::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 08:13:17,423::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 08:13:27,424::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 08:13:27,425::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 08:13:27,425::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110524::WARNING::2014-10-05 08:14:27,425::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 08:14:27,425::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 08:14:27,425::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 08:14:37,427::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 08:14:37,427::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110525::WARNING::2014-10-05 08:15:37,427::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 08:15:37,427::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 08:15:37,427::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 08:15:47,429::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 08:15:47,429::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110526::WARNING::2014-10-05 08:16:47,429::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 08:16:47,429::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 08:16:47,429::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 08:16:57,431::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 08:16:57,431::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-05 08:17:52,732::__init__::232::IOProcess::(_processLogs) (92323) Queuing response >Thread-18::DEBUG::2014-10-05 08:17:52,733::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 08:17:52,733::__init__::232::IOProcess::(_processLogs) (92383) Got request for method 'glob' >Thread-110527::WARNING::2014-10-05 08:17:57,431::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 08:17:57,431::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 08:17:57,431::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 08:18:07,433::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 08:18:07,433::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110528::WARNING::2014-10-05 08:19:07,433::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 08:19:07,433::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 08:19:07,433::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 08:19:17,434::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 08:19:17,435::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 08:19:17,435::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110529::WARNING::2014-10-05 08:20:17,435::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 08:20:17,435::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 08:20:17,436::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 08:20:27,437::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 08:20:27,437::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110530::WARNING::2014-10-05 08:21:27,437::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 08:21:27,437::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 08:21:27,438::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 08:21:37,439::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 08:21:37,439::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110531::WARNING::2014-10-05 08:22:37,439::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 08:22:37,439::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 08:22:37,439::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 08:22:47,441::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 08:22:47,441::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110532::WARNING::2014-10-05 08:23:47,440::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 08:23:47,441::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 08:23:47,441::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 08:23:57,442::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 08:23:57,443::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110533::WARNING::2014-10-05 08:24:57,442::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 08:24:57,443::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 08:24:57,443::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 08:25:07,443::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 08:25:07,444::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 08:25:07,444::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110534::WARNING::2014-10-05 08:26:07,444::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 08:26:07,445::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 08:26:07,445::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 08:26:17,446::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 08:26:17,446::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110535::WARNING::2014-10-05 08:27:17,446::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 08:27:17,446::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 08:27:17,447::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 08:27:27,448::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 08:27:27,448::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110536::WARNING::2014-10-05 08:28:27,448::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 08:28:27,448::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 08:28:27,448::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 08:28:37,449::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 08:28:37,450::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110537::WARNING::2014-10-05 08:29:37,449::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 08:29:37,450::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 08:29:37,450::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 08:29:47,451::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 08:29:47,451::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110538::WARNING::2014-10-05 08:30:47,451::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 08:30:47,452::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 08:30:47,452::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 08:30:57,452::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 08:30:57,453::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 08:30:57,453::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110539::WARNING::2014-10-05 08:31:57,453::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 08:31:57,454::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 08:31:57,454::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 08:32:07,455::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 08:32:07,455::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110540::WARNING::2014-10-05 08:33:07,455::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 08:33:07,455::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 08:33:07,456::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 08:33:17,457::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 08:33:17,457::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110541::WARNING::2014-10-05 08:34:17,457::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 08:34:17,457::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 08:34:17,457::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 08:34:27,458::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 08:34:27,459::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110542::WARNING::2014-10-05 08:35:27,458::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 08:35:27,459::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 08:35:27,459::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 08:35:37,460::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 08:35:37,460::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110543::WARNING::2014-10-05 08:36:37,460::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 08:36:37,461::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 08:36:37,461::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 08:36:47,461::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 08:36:47,462::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 08:36:47,462::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110544::WARNING::2014-10-05 08:37:47,462::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 08:37:47,463::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 08:37:47,463::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 08:37:57,464::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 08:37:57,464::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110545::WARNING::2014-10-05 08:38:57,464::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 08:38:57,464::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 08:38:57,465::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 08:39:07,466::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 08:39:07,466::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110546::WARNING::2014-10-05 08:40:07,466::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 08:40:07,466::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 08:40:07,466::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 08:40:17,467::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 08:40:17,468::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110547::WARNING::2014-10-05 08:41:17,467::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 08:41:17,468::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 08:41:17,468::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 08:41:27,469::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 08:41:27,469::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110548::WARNING::2014-10-05 08:42:27,469::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 08:42:27,470::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 08:42:27,470::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 08:42:37,470::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 08:42:37,471::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 08:42:37,471::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110549::WARNING::2014-10-05 08:43:37,471::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 08:43:37,472::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 08:43:37,472::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 08:43:47,473::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 08:43:47,473::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110550::WARNING::2014-10-05 08:44:47,473::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 08:44:47,473::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 08:44:47,474::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 08:44:57,475::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 08:44:57,475::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110551::WARNING::2014-10-05 08:45:57,475::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 08:45:57,475::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 08:45:57,475::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 08:46:07,477::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 08:46:07,477::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110552::WARNING::2014-10-05 08:47:07,476::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 08:47:07,477::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 08:47:07,477::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 08:47:17,478::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 08:47:17,479::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110553::WARNING::2014-10-05 08:48:17,478::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 08:48:17,479::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 08:48:17,479::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 08:48:27,479::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 08:48:27,480::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 08:48:27,480::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110554::WARNING::2014-10-05 08:49:27,480::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 08:49:27,481::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 08:49:27,481::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 08:49:37,483::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 08:49:37,484::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110555::WARNING::2014-10-05 08:50:37,483::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 08:50:37,484::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 08:50:37,484::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 08:50:47,485::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 08:50:47,486::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110556::WARNING::2014-10-05 08:51:47,485::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 08:51:47,486::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 08:51:47,486::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 08:51:57,487::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 08:51:57,488::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110557::WARNING::2014-10-05 08:52:57,487::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 08:52:57,488::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 08:52:57,488::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 08:53:07,489::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 08:53:07,489::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110558::WARNING::2014-10-05 08:54:07,489::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 08:54:07,490::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 08:54:07,490::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 08:54:17,490::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 08:54:17,491::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 08:54:17,491::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110559::WARNING::2014-10-05 08:55:17,491::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 08:55:17,492::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 08:55:17,492::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 08:55:27,493::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 08:55:27,493::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110560::WARNING::2014-10-05 08:56:27,493::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 08:56:27,494::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 08:56:27,494::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 08:56:37,495::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 08:56:37,495::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110561::WARNING::2014-10-05 08:57:37,495::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 08:57:37,496::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 08:57:37,496::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 08:57:47,497::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 08:57:47,497::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110562::WARNING::2014-10-05 08:58:47,497::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 08:58:47,498::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 08:58:47,498::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 08:58:57,499::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 08:58:57,499::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110563::WARNING::2014-10-05 08:59:57,499::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 08:59:57,500::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 08:59:57,500::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 09:00:07,500::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 09:00:07,501::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 09:00:07,501::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110564::WARNING::2014-10-05 09:01:07,501::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 09:01:07,503::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 09:01:07,503::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 09:01:17,505::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 09:01:17,506::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110565::WARNING::2014-10-05 09:02:17,505::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 09:02:17,506::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 09:02:17,506::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 09:02:27,507::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 09:02:27,507::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110566::WARNING::2014-10-05 09:03:27,507::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 09:03:27,508::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 09:03:27,508::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 09:03:37,509::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 09:03:37,509::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110567::WARNING::2014-10-05 09:04:37,509::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 09:04:37,510::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 09:04:37,510::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 09:04:47,511::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 09:04:47,511::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110568::WARNING::2014-10-05 09:05:47,511::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 09:05:47,512::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 09:05:47,512::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 09:05:57,512::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 09:05:57,513::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 09:05:57,513::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110569::WARNING::2014-10-05 09:06:57,513::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 09:06:57,514::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 09:06:57,514::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 09:07:07,515::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 09:07:07,515::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110570::WARNING::2014-10-05 09:08:07,515::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 09:08:07,515::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 09:08:07,516::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 09:08:17,517::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 09:08:17,518::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110571::WARNING::2014-10-05 09:09:17,517::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 09:09:17,518::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 09:09:17,518::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 09:09:27,519::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 09:09:27,520::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110572::WARNING::2014-10-05 09:10:27,519::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 09:10:27,520::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 09:10:27,520::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 09:10:37,521::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 09:10:37,521::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110573::WARNING::2014-10-05 09:11:37,521::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 09:11:37,522::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 09:11:37,522::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 09:11:47,522::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 09:11:47,523::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 09:11:47,523::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110574::WARNING::2014-10-05 09:12:47,523::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 09:12:47,524::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 09:12:47,524::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 09:12:57,525::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 09:12:57,525::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110575::WARNING::2014-10-05 09:13:57,525::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 09:13:57,525::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 09:13:57,526::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 09:14:07,527::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 09:14:07,527::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110576::WARNING::2014-10-05 09:15:07,527::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 09:15:07,527::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 09:15:07,527::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 09:15:17,529::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 09:15:17,529::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110577::WARNING::2014-10-05 09:16:17,529::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 09:16:17,529::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 09:16:17,529::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 09:16:27,531::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 09:16:27,531::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110578::WARNING::2014-10-05 09:17:27,530::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 09:17:27,531::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 09:17:27,531::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 09:17:37,532::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 09:17:37,533::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 09:17:37,533::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110579::WARNING::2014-10-05 09:18:37,533::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 09:18:37,533::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 09:18:37,533::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 09:18:47,535::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 09:18:47,535::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110580::WARNING::2014-10-05 09:19:47,535::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 09:19:47,535::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 09:19:47,535::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 09:19:57,537::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 09:19:57,537::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110581::WARNING::2014-10-05 09:20:57,537::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 09:20:57,537::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 09:20:57,537::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 09:21:07,539::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 09:21:07,539::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110582::WARNING::2014-10-05 09:22:07,539::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 09:22:07,539::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 09:22:07,539::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 09:22:17,541::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 09:22:17,541::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110583::WARNING::2014-10-05 09:23:17,540::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 09:23:17,541::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 09:23:17,541::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 09:23:27,541::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 09:23:27,542::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 09:23:27,543::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110584::WARNING::2014-10-05 09:24:27,542::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 09:24:27,543::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 09:24:27,543::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 09:24:37,544::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 09:24:37,544::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110585::WARNING::2014-10-05 09:25:37,544::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 09:25:37,545::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 09:25:37,545::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 09:25:47,546::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 09:25:47,546::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110586::WARNING::2014-10-05 09:26:47,546::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 09:26:47,546::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 09:26:47,547::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 09:26:57,548::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 09:26:57,548::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110587::WARNING::2014-10-05 09:27:57,548::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 09:27:57,548::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 09:27:57,548::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 09:28:07,549::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 09:28:07,550::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110588::WARNING::2014-10-05 09:29:07,549::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 09:29:07,550::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 09:29:07,550::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 09:29:17,550::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 09:29:17,551::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 09:29:17,552::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110589::WARNING::2014-10-05 09:30:17,551::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 09:30:17,552::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 09:30:17,552::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 09:30:27,553::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 09:30:27,553::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110590::WARNING::2014-10-05 09:31:27,553::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 09:31:27,554::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 09:31:27,554::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 09:31:37,555::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 09:31:37,555::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110591::WARNING::2014-10-05 09:32:37,555::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 09:32:37,555::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 09:32:37,556::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 09:32:47,557::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 09:32:47,557::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110592::WARNING::2014-10-05 09:33:47,557::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 09:33:47,557::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 09:33:47,557::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 09:33:57,558::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 09:33:57,559::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110593::WARNING::2014-10-05 09:34:57,558::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 09:34:57,559::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 09:34:57,559::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 09:35:07,559::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 09:35:07,560::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 09:35:07,560::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110594::WARNING::2014-10-05 09:36:07,560::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 09:36:07,561::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 09:36:07,561::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 09:36:17,562::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 09:36:17,562::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110595::WARNING::2014-10-05 09:37:17,562::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 09:37:17,562::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 09:37:17,563::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 09:37:27,564::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 09:37:27,564::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-05 09:38:26,012::__init__::232::IOProcess::(_processLogs) (92324) Queuing response >Thread-18::DEBUG::2014-10-05 09:38:26,013::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 09:38:26,013::__init__::232::IOProcess::(_processLogs) (92384) Got request for method 'glob' >Thread-110596::WARNING::2014-10-05 09:38:27,564::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 09:38:27,564::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 09:38:27,564::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 09:38:37,565::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 09:38:37,566::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110597::WARNING::2014-10-05 09:39:37,565::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 09:39:37,566::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 09:39:37,566::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 09:39:47,567::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 09:39:47,567::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110598::WARNING::2014-10-05 09:40:47,567::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 09:40:47,567::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 09:40:47,568::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 09:40:57,568::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 09:40:57,569::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 09:40:57,569::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110599::WARNING::2014-10-05 09:41:57,569::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 09:41:57,569::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 09:41:57,570::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 09:42:07,571::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 09:42:07,571::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110600::WARNING::2014-10-05 09:43:07,571::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 09:43:07,571::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 09:43:07,572::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 09:43:17,573::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 09:43:17,573::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110601::WARNING::2014-10-05 09:44:17,573::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 09:44:17,573::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 09:44:17,573::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 09:44:27,574::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 09:44:27,575::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110602::WARNING::2014-10-05 09:45:27,574::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 09:45:27,575::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 09:45:27,575::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 09:45:37,576::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 09:45:37,577::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110603::WARNING::2014-10-05 09:46:37,576::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 09:46:37,577::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 09:46:37,577::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 09:46:47,577::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 09:46:47,578::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 09:46:47,578::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110604::WARNING::2014-10-05 09:47:47,578::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 09:47:47,579::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 09:47:47,579::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 09:47:57,580::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 09:47:57,580::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110605::WARNING::2014-10-05 09:48:57,580::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 09:48:57,581::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 09:48:57,581::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 09:49:07,582::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 09:49:07,583::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110606::WARNING::2014-10-05 09:50:07,582::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 09:50:07,583::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 09:50:07,583::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 09:50:17,584::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 09:50:17,585::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110607::WARNING::2014-10-05 09:51:17,584::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 09:51:17,585::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 09:51:17,585::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 09:51:27,586::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 09:51:27,586::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110608::WARNING::2014-10-05 09:52:27,586::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 09:52:27,587::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 09:52:27,587::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 09:52:37,587::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 09:52:37,588::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 09:52:37,588::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110609::WARNING::2014-10-05 09:53:37,588::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 09:53:37,589::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 09:53:37,589::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 09:53:47,590::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 09:53:47,590::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110610::WARNING::2014-10-05 09:54:47,590::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 09:54:47,591::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 09:54:47,591::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 09:54:57,592::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 09:54:57,592::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110611::WARNING::2014-10-05 09:55:57,592::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 09:55:57,593::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 09:55:57,593::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 09:56:07,594::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 09:56:07,594::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110612::WARNING::2014-10-05 09:57:07,594::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 09:57:07,594::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 09:57:07,595::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 09:57:17,596::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 09:57:17,596::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110613::WARNING::2014-10-05 09:58:17,596::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 09:58:17,596::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 09:58:17,597::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 09:58:27,597::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 09:58:27,598::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 09:58:27,598::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110614::WARNING::2014-10-05 09:59:27,598::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 09:59:27,598::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 09:59:27,598::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 09:59:37,600::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 09:59:37,600::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110615::WARNING::2014-10-05 10:00:37,600::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 10:00:37,600::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 10:00:37,600::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 10:00:47,601::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 10:00:47,602::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110616::WARNING::2014-10-05 10:01:47,601::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 10:01:47,602::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 10:01:47,602::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 10:01:57,603::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 10:01:57,604::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110617::WARNING::2014-10-05 10:02:57,603::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 10:02:57,604::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 10:02:57,604::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 10:03:07,605::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 10:03:07,605::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110618::WARNING::2014-10-05 10:04:07,605::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 10:04:07,606::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 10:04:07,606::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 10:04:17,606::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 10:04:17,607::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 10:04:17,607::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110619::WARNING::2014-10-05 10:05:17,607::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 10:05:17,608::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 10:05:17,608::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 10:05:27,609::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 10:05:27,609::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110620::WARNING::2014-10-05 10:06:27,609::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 10:06:27,610::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 10:06:27,610::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 10:06:37,611::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 10:06:37,611::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110621::WARNING::2014-10-05 10:07:37,611::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 10:07:37,611::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 10:07:37,612::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 10:07:47,613::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 10:07:47,613::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110622::WARNING::2014-10-05 10:08:47,613::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 10:08:47,613::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 10:08:47,614::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 10:08:57,615::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 10:08:57,615::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110623::WARNING::2014-10-05 10:09:57,614::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 10:09:57,615::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 10:09:57,615::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 10:10:07,616::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 10:10:07,617::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 10:10:07,617::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110624::WARNING::2014-10-05 10:11:07,617::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 10:11:07,617::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 10:11:07,617::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 10:11:17,618::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 10:11:17,619::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110625::WARNING::2014-10-05 10:12:17,618::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 10:12:17,619::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 10:12:17,619::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 10:12:27,620::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 10:12:27,620::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110626::WARNING::2014-10-05 10:13:27,620::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 10:13:27,621::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 10:13:27,621::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 10:13:37,622::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 10:13:37,622::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110627::WARNING::2014-10-05 10:14:37,622::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 10:14:37,622::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 10:14:37,623::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 10:14:47,624::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 10:14:47,624::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110628::WARNING::2014-10-05 10:15:47,624::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 10:15:47,624::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 10:15:47,624::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 10:15:57,625::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 10:15:57,626::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 10:15:57,626::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110629::WARNING::2014-10-05 10:16:57,626::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 10:16:57,626::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 10:16:57,626::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 10:17:07,628::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 10:17:07,628::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110630::WARNING::2014-10-05 10:18:07,628::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 10:18:07,628::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 10:18:07,628::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 10:18:17,630::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 10:18:17,630::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-05 10:18:44,701::__init__::232::IOProcess::(_processLogs) (92325) Queuing response >Thread-18::DEBUG::2014-10-05 10:18:44,701::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 10:18:44,701::__init__::232::IOProcess::(_processLogs) (92385) Got request for method 'glob' >Thread-110631::WARNING::2014-10-05 10:19:17,630::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 10:19:17,630::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 10:19:17,630::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 10:19:27,631::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 10:19:27,632::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110632::WARNING::2014-10-05 10:20:27,631::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 10:20:27,632::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 10:20:27,632::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 10:20:37,633::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 10:20:37,634::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110633::WARNING::2014-10-05 10:21:37,633::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 10:21:37,634::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 10:21:37,634::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 10:21:47,635::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 10:21:47,636::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 10:21:47,636::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110634::WARNING::2014-10-05 10:22:47,636::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 10:22:47,636::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 10:22:47,636::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 10:22:57,638::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 10:22:57,638::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110635::WARNING::2014-10-05 10:23:57,637::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 10:23:57,638::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 10:23:57,638::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 10:24:07,639::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 10:24:07,639::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110636::WARNING::2014-10-05 10:25:07,639::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 10:25:07,640::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 10:25:07,640::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 10:25:17,641::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 10:25:17,641::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110637::WARNING::2014-10-05 10:26:17,641::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 10:26:17,642::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 10:26:17,642::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 10:26:27,643::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 10:26:27,643::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110638::WARNING::2014-10-05 10:27:27,643::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 10:27:27,644::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 10:27:27,644::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 10:27:37,644::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 10:27:37,645::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 10:27:37,645::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110639::WARNING::2014-10-05 10:28:37,645::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 10:28:37,646::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 10:28:37,646::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 10:28:47,647::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 10:28:47,647::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-05 10:28:48,860::__init__::232::IOProcess::(_processLogs) (92326) Queuing response >Thread-18::DEBUG::2014-10-05 10:28:48,861::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 10:28:48,861::__init__::232::IOProcess::(_processLogs) (92386) Got request for method 'glob' >Thread-110640::WARNING::2014-10-05 10:29:47,647::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 10:29:47,648::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 10:29:47,648::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 10:29:57,649::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 10:29:57,649::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110641::WARNING::2014-10-05 10:30:57,649::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 10:30:57,649::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 10:30:57,650::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 10:31:07,651::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 10:31:07,651::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110642::WARNING::2014-10-05 10:32:07,651::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 10:32:07,651::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 10:32:07,651::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 10:32:17,653::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 10:32:17,653::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110643::WARNING::2014-10-05 10:33:17,652::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 10:33:17,653::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 10:33:17,653::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 10:33:27,654::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 10:33:27,655::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 10:33:27,655::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110644::WARNING::2014-10-05 10:34:27,655::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 10:34:27,655::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 10:34:27,655::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 10:34:37,656::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 10:34:37,657::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110645::WARNING::2014-10-05 10:35:37,656::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 10:35:37,657::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 10:35:37,657::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 10:35:47,658::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 10:35:47,658::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110646::WARNING::2014-10-05 10:36:47,658::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 10:36:47,658::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 10:36:47,659::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 10:36:57,660::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 10:36:57,660::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110647::WARNING::2014-10-05 10:37:57,660::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 10:37:57,660::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 10:37:57,660::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 10:38:07,662::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 10:38:07,662::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-05 10:38:53,020::__init__::232::IOProcess::(_processLogs) (92327) Queuing response >Thread-18::DEBUG::2014-10-05 10:38:53,021::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 10:38:53,021::__init__::232::IOProcess::(_processLogs) (92387) Got request for method 'glob' >Thread-110648::WARNING::2014-10-05 10:39:07,662::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 10:39:07,662::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 10:39:07,662::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 10:39:17,663::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 10:39:17,664::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 10:39:17,664::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110649::WARNING::2014-10-05 10:40:17,664::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 10:40:17,664::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 10:40:17,664::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 10:40:27,665::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 10:40:27,666::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110650::WARNING::2014-10-05 10:41:27,665::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 10:41:27,666::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 10:41:27,666::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 10:41:37,667::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 10:41:37,668::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110651::WARNING::2014-10-05 10:42:37,667::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 10:42:37,668::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 10:42:37,668::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 10:42:47,669::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 10:42:47,670::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110652::WARNING::2014-10-05 10:43:47,669::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 10:43:47,670::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 10:43:47,670::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 10:43:57,671::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 10:43:57,671::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110653::WARNING::2014-10-05 10:44:57,671::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 10:44:57,672::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 10:44:57,672::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 10:45:07,672::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 10:45:07,673::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 10:45:07,673::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110654::WARNING::2014-10-05 10:46:07,673::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 10:46:07,674::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 10:46:07,674::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 10:46:17,675::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 10:46:17,675::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110655::WARNING::2014-10-05 10:47:17,675::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 10:47:17,676::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 10:47:17,676::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 10:47:27,677::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 10:47:27,677::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110656::WARNING::2014-10-05 10:48:27,677::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 10:48:27,678::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 10:48:27,678::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 10:48:37,679::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 10:48:37,679::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110657::WARNING::2014-10-05 10:49:37,679::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 10:49:37,680::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 10:49:37,680::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 10:49:47,681::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 10:49:47,681::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110658::WARNING::2014-10-05 10:50:47,681::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 10:50:47,682::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 10:50:47,682::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 10:50:57,682::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 10:50:57,683::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 10:50:57,683::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110659::WARNING::2014-10-05 10:51:57,683::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 10:51:57,684::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 10:51:57,684::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 10:52:07,685::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 10:52:07,685::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110660::WARNING::2014-10-05 10:53:07,685::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 10:53:07,686::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 10:53:07,686::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 10:53:17,687::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 10:53:17,687::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110661::WARNING::2014-10-05 10:54:17,687::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 10:54:17,688::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 10:54:17,688::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 10:54:27,689::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 10:54:27,689::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110662::WARNING::2014-10-05 10:55:27,689::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 10:55:27,689::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 10:55:27,690::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 10:55:37,691::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 10:55:37,691::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110663::WARNING::2014-10-05 10:56:37,691::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 10:56:37,691::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 10:56:37,692::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 10:56:47,692::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 10:56:47,693::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 10:56:47,693::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110664::WARNING::2014-10-05 10:57:47,693::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 10:57:47,693::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 10:57:47,694::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 10:57:57,695::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 10:57:57,695::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110665::WARNING::2014-10-05 10:58:57,695::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 10:58:57,695::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 10:58:57,696::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 10:59:07,697::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 10:59:07,697::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110666::WARNING::2014-10-05 11:00:07,697::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 11:00:07,697::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 11:00:07,697::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 11:00:17,699::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 11:00:17,699::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110667::WARNING::2014-10-05 11:01:17,698::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 11:01:17,699::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 11:01:17,699::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 11:01:27,701::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 11:01:27,701::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110668::WARNING::2014-10-05 11:02:27,700::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 11:02:27,701::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 11:02:27,701::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 11:02:37,702::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 11:02:37,703::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 11:02:37,703::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110669::WARNING::2014-10-05 11:03:37,703::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 11:03:37,703::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 11:03:37,703::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 11:03:47,705::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 11:03:47,705::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110670::WARNING::2014-10-05 11:04:47,704::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 11:04:47,705::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 11:04:47,705::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 11:04:57,706::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 11:04:57,706::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110671::WARNING::2014-10-05 11:05:57,706::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 11:05:57,708::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 11:05:57,708::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 11:06:07,710::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 11:06:07,710::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110672::WARNING::2014-10-05 11:07:07,710::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 11:07:07,710::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 11:07:07,710::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 11:07:17,712::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 11:07:17,712::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110673::WARNING::2014-10-05 11:08:17,711::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 11:08:17,712::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 11:08:17,712::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 11:08:27,713::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 11:08:27,713::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 11:08:27,714::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110674::WARNING::2014-10-05 11:09:27,713::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 11:09:27,714::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 11:09:27,714::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 11:09:37,715::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 11:09:37,716::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110675::WARNING::2014-10-05 11:10:37,715::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 11:10:37,716::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 11:10:37,716::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 11:10:47,717::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 11:10:47,718::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110676::WARNING::2014-10-05 11:11:47,717::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 11:11:47,718::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 11:11:47,718::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 11:11:57,719::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 11:11:57,719::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110677::WARNING::2014-10-05 11:12:57,719::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 11:12:57,720::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 11:12:57,720::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 11:13:07,721::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 11:13:07,721::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110678::WARNING::2014-10-05 11:14:07,721::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 11:14:07,721::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 11:14:07,722::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 11:14:17,722::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 11:14:17,723::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 11:14:17,723::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110679::WARNING::2014-10-05 11:15:17,723::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 11:15:17,723::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 11:15:17,724::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 11:15:27,725::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 11:15:27,725::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110680::WARNING::2014-10-05 11:16:27,725::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 11:16:27,725::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 11:16:27,726::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 11:16:37,727::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 11:16:37,727::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110681::WARNING::2014-10-05 11:17:37,727::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 11:17:37,727::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 11:17:37,727::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 11:17:47,729::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 11:17:47,729::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110682::WARNING::2014-10-05 11:18:47,728::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 11:18:47,729::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 11:18:47,729::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 11:18:57,730::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 11:18:57,730::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110683::WARNING::2014-10-05 11:19:57,730::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 11:19:57,731::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 11:19:57,731::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 11:20:07,731::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 11:20:07,732::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 11:20:07,733::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110684::WARNING::2014-10-05 11:21:07,732::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 11:21:07,733::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 11:21:07,733::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 11:21:17,734::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 11:21:17,734::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110685::WARNING::2014-10-05 11:22:17,734::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 11:22:17,735::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 11:22:17,735::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 11:22:27,736::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 11:22:27,736::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110686::WARNING::2014-10-05 11:23:27,736::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 11:23:27,736::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 11:23:27,737::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 11:23:37,738::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 11:23:37,738::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110687::WARNING::2014-10-05 11:24:37,738::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 11:24:37,738::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 11:24:37,739::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 11:24:47,740::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 11:24:47,740::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110688::WARNING::2014-10-05 11:25:47,740::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 11:25:47,740::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 11:25:47,740::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 11:25:57,741::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 11:25:57,741::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 11:25:57,742::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110689::WARNING::2014-10-05 11:26:57,741::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 11:26:57,742::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 11:26:57,742::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 11:27:07,743::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 11:27:07,743::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110690::WARNING::2014-10-05 11:28:07,743::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 11:28:07,744::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 11:28:07,744::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 11:28:17,745::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 11:28:17,745::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-05 11:29:13,820::__init__::232::IOProcess::(_processLogs) (92328) Queuing response >Thread-18::DEBUG::2014-10-05 11:29:13,821::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 11:29:13,821::__init__::232::IOProcess::(_processLogs) (92388) Got request for method 'glob' >Thread-110691::WARNING::2014-10-05 11:29:17,745::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 11:29:17,745::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 11:29:17,746::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 11:29:27,747::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 11:29:27,747::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110692::WARNING::2014-10-05 11:30:27,747::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 11:30:27,747::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 11:30:27,747::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 11:30:37,748::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 11:30:37,749::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110693::WARNING::2014-10-05 11:31:37,748::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 11:31:37,749::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 11:31:37,749::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 11:31:47,749::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 11:31:47,750::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 11:31:47,750::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110694::WARNING::2014-10-05 11:32:47,750::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 11:32:47,751::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 11:32:47,751::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 11:32:57,752::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 11:32:57,752::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110695::WARNING::2014-10-05 11:33:57,752::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 11:33:57,752::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 11:33:57,753::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 11:34:07,754::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 11:34:07,754::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110696::WARNING::2014-10-05 11:35:07,754::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 11:35:07,754::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 11:35:07,754::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 11:35:17,756::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 11:35:17,756::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110697::WARNING::2014-10-05 11:36:17,756::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 11:36:17,756::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 11:36:17,756::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 11:36:27,757::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 11:36:27,758::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110698::WARNING::2014-10-05 11:37:27,757::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 11:37:27,758::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 11:37:27,758::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 11:37:37,759::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 11:37:37,760::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 11:37:37,760::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110699::WARNING::2014-10-05 11:38:37,759::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 11:38:37,760::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 11:38:37,760::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 11:38:47,761::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 11:38:47,762::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-05 11:39:17,980::__init__::232::IOProcess::(_processLogs) (92329) Queuing response >Thread-18::DEBUG::2014-10-05 11:39:17,981::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 11:39:17,981::__init__::232::IOProcess::(_processLogs) (92389) Got request for method 'glob' >Thread-110700::WARNING::2014-10-05 11:39:47,761::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 11:39:47,762::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 11:39:47,762::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 11:39:57,764::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 11:39:57,764::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110701::WARNING::2014-10-05 11:40:57,763::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 11:40:57,764::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 11:40:57,764::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 11:41:07,765::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 11:41:07,766::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110702::WARNING::2014-10-05 11:42:07,765::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 11:42:07,766::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 11:42:07,766::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 11:42:17,767::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 11:42:17,768::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110703::WARNING::2014-10-05 11:43:17,767::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 11:43:17,768::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 11:43:17,768::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 11:43:27,768::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 11:43:27,770::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 11:43:27,770::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110704::WARNING::2014-10-05 11:44:27,770::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 11:44:27,770::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 11:44:27,770::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 11:44:37,771::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 11:44:37,772::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110705::WARNING::2014-10-05 11:45:37,771::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 11:45:37,772::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 11:45:37,772::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 11:45:47,773::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 11:45:47,774::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110706::WARNING::2014-10-05 11:46:47,773::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 11:46:47,774::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 11:46:47,774::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 11:46:57,775::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 11:46:57,775::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110707::WARNING::2014-10-05 11:47:57,775::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 11:47:57,776::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 11:47:57,776::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 11:48:07,777::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 11:48:07,777::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110708::WARNING::2014-10-05 11:49:07,777::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 11:49:07,778::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 11:49:07,778::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 11:49:17,778::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 11:49:17,779::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 11:49:17,779::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110709::WARNING::2014-10-05 11:50:17,779::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 11:50:17,780::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 11:50:17,780::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 11:50:27,781::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 11:50:27,781::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110710::WARNING::2014-10-05 11:51:27,781::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 11:51:27,782::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 11:51:27,782::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 11:51:37,783::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 11:51:37,783::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110711::WARNING::2014-10-05 11:52:37,783::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 11:52:37,783::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 11:52:37,784::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 11:52:47,785::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 11:52:47,785::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110712::WARNING::2014-10-05 11:53:47,785::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 11:53:47,785::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 11:53:47,786::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 11:53:57,787::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 11:53:57,787::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110713::WARNING::2014-10-05 11:54:57,787::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 11:54:57,787::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 11:54:57,787::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 11:55:07,788::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 11:55:07,789::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 11:55:07,789::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110714::WARNING::2014-10-05 11:56:07,789::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 11:56:07,789::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 11:56:07,789::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 11:56:17,791::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 11:56:17,791::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110715::WARNING::2014-10-05 11:57:17,791::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 11:57:17,791::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 11:57:17,791::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 11:57:27,793::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 11:57:27,793::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110716::WARNING::2014-10-05 11:58:27,793::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 11:58:27,793::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 11:58:27,793::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 11:58:37,794::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 11:58:37,795::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110717::WARNING::2014-10-05 11:59:37,794::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 11:59:37,795::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 11:59:37,795::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 11:59:47,796::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 11:59:47,797::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110718::WARNING::2014-10-05 12:00:47,796::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 12:00:47,797::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 12:00:47,797::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 12:00:57,797::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 12:00:57,798::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 12:00:57,799::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110719::WARNING::2014-10-05 12:01:57,798::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 12:01:57,799::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 12:01:57,799::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 12:02:07,800::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 12:02:07,801::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110720::WARNING::2014-10-05 12:03:07,800::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 12:03:07,801::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 12:03:07,801::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 12:03:17,802::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 12:03:17,803::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110721::WARNING::2014-10-05 12:04:17,802::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 12:04:17,803::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 12:04:17,803::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 12:04:27,804::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 12:04:27,805::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110722::WARNING::2014-10-05 12:05:27,804::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 12:05:27,805::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 12:05:27,805::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 12:05:37,806::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 12:05:37,806::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110723::WARNING::2014-10-05 12:06:37,806::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 12:06:37,807::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 12:06:37,807::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 12:06:47,807::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 12:06:47,808::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 12:06:47,808::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110724::WARNING::2014-10-05 12:07:47,808::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 12:07:47,809::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 12:07:47,809::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 12:07:57,810::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 12:07:57,810::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110725::WARNING::2014-10-05 12:08:57,810::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 12:08:57,811::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 12:08:57,811::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 12:09:07,812::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 12:09:07,812::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110726::WARNING::2014-10-05 12:10:07,812::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 12:10:07,812::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 12:10:07,813::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 12:10:17,814::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 12:10:17,814::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110727::WARNING::2014-10-05 12:11:17,814::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 12:11:17,814::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 12:11:17,815::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 12:11:27,816::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 12:11:27,816::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110728::WARNING::2014-10-05 12:12:27,816::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 12:12:27,816::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 12:12:27,816::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 12:12:37,817::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 12:12:37,818::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 12:12:37,818::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110729::WARNING::2014-10-05 12:13:37,818::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 12:13:37,818::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 12:13:37,818::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 12:13:47,820::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 12:13:47,820::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110730::WARNING::2014-10-05 12:14:47,820::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 12:14:47,820::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 12:14:47,820::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 12:14:57,822::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 12:14:57,822::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110731::WARNING::2014-10-05 12:15:57,821::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 12:15:57,822::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 12:15:57,822::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 12:16:07,823::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 12:16:07,824::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110732::WARNING::2014-10-05 12:17:07,823::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 12:17:07,824::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 12:17:07,824::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 12:17:17,825::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 12:17:17,826::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110733::WARNING::2014-10-05 12:18:17,825::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 12:18:17,826::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 12:18:17,826::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 12:18:27,826::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 12:18:27,827::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 12:18:27,828::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110734::WARNING::2014-10-05 12:19:27,827::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 12:19:27,828::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 12:19:27,828::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 12:19:37,829::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 12:19:37,830::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110735::WARNING::2014-10-05 12:20:37,829::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 12:20:37,830::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 12:20:37,830::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 12:20:47,831::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 12:20:47,832::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110736::WARNING::2014-10-05 12:21:47,831::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 12:21:47,832::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 12:21:47,832::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 12:21:57,834::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 12:21:57,834::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110737::WARNING::2014-10-05 12:22:57,833::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 12:22:57,834::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 12:22:57,834::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 12:23:07,835::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 12:23:07,835::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110738::WARNING::2014-10-05 12:24:07,835::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 12:24:07,836::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 12:24:07,836::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 12:24:17,836::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 12:24:17,837::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 12:24:17,837::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110739::WARNING::2014-10-05 12:25:17,837::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 12:25:17,837::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 12:25:17,838::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 12:25:27,839::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 12:25:27,839::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110740::WARNING::2014-10-05 12:26:27,839::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 12:26:27,839::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 12:26:27,840::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 12:26:37,841::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 12:26:37,841::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110741::WARNING::2014-10-05 12:27:37,841::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 12:27:37,841::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 12:27:37,841::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 12:27:47,842::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 12:27:47,843::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110742::WARNING::2014-10-05 12:28:47,842::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 12:28:47,843::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 12:28:47,843::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 12:28:57,844::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 12:28:57,845::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110743::WARNING::2014-10-05 12:29:57,844::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 12:29:57,845::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 12:29:57,845::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 12:30:07,845::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 12:30:07,846::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 12:30:07,847::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110744::WARNING::2014-10-05 12:31:07,846::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 12:31:07,847::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 12:31:07,847::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 12:31:17,848::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 12:31:17,849::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110745::WARNING::2014-10-05 12:32:17,848::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 12:32:17,849::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 12:32:17,849::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 12:32:27,850::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 12:32:27,850::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110746::WARNING::2014-10-05 12:33:27,850::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 12:33:27,851::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 12:33:27,851::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 12:33:37,852::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 12:33:37,852::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110747::WARNING::2014-10-05 12:34:37,852::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 12:34:37,853::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 12:34:37,853::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 12:34:47,854::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 12:34:47,854::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110748::WARNING::2014-10-05 12:35:47,854::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 12:35:47,854::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 12:35:47,855::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 12:35:57,855::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 12:35:57,856::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 12:35:57,856::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110749::WARNING::2014-10-05 12:36:57,856::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 12:36:57,857::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 12:36:57,857::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 12:37:07,858::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 12:37:07,858::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110750::WARNING::2014-10-05 12:38:07,858::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 12:38:07,859::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 12:38:07,859::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 12:38:17,860::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 12:38:17,860::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110751::WARNING::2014-10-05 12:39:17,860::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 12:39:17,860::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 12:39:17,861::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 12:39:27,862::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 12:39:27,862::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110752::WARNING::2014-10-05 12:40:27,862::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 12:40:27,862::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 12:40:27,863::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 12:40:37,864::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 12:40:37,864::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110753::WARNING::2014-10-05 12:41:37,864::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 12:41:37,864::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 12:41:37,864::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 12:41:47,865::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 12:41:47,866::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 12:41:47,866::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110754::WARNING::2014-10-05 12:42:47,866::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 12:42:47,866::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 12:42:47,866::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 12:42:57,867::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 12:42:57,868::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110755::WARNING::2014-10-05 12:43:57,867::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 12:43:57,868::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 12:43:57,868::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 12:44:07,869::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 12:44:07,870::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110756::WARNING::2014-10-05 12:45:07,869::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 12:45:07,870::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 12:45:07,870::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 12:45:17,871::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 12:45:17,872::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110757::WARNING::2014-10-05 12:46:17,871::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 12:46:17,872::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 12:46:17,872::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 12:46:27,873::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 12:46:27,874::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110758::WARNING::2014-10-05 12:47:27,873::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 12:47:27,874::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 12:47:27,874::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 12:47:37,875::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 12:47:37,876::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 12:47:37,876::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110759::WARNING::2014-10-05 12:48:37,876::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 12:48:37,876::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 12:48:37,876::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 12:48:47,878::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 12:48:47,878::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110760::WARNING::2014-10-05 12:49:47,878::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 12:49:47,878::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 12:49:47,878::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 12:49:57,880::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 12:49:57,880::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110761::WARNING::2014-10-05 12:50:57,880::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 12:50:57,880::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 12:50:57,880::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 12:51:07,882::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 12:51:07,882::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110762::WARNING::2014-10-05 12:52:07,882::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 12:52:07,882::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 12:52:07,882::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 12:52:17,885::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 12:52:17,885::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110763::WARNING::2014-10-05 12:53:17,885::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 12:53:17,885::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 12:53:17,886::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 12:53:27,886::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 12:53:27,887::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 12:53:27,887::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110764::WARNING::2014-10-05 12:54:27,887::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 12:54:27,887::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 12:54:27,888::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 12:54:37,889::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 12:54:37,889::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110765::WARNING::2014-10-05 12:55:37,889::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 12:55:37,889::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 12:55:37,890::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 12:55:47,891::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 12:55:47,891::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110766::WARNING::2014-10-05 12:56:47,891::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 12:56:47,891::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 12:56:47,892::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 12:56:57,893::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 12:56:57,893::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110767::WARNING::2014-10-05 12:57:57,893::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 12:57:57,893::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 12:57:57,894::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 12:58:07,895::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 12:58:07,895::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110768::WARNING::2014-10-05 12:59:07,895::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 12:59:07,895::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 12:59:07,895::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 12:59:17,896::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 12:59:17,897::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 12:59:17,897::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-05 12:59:51,260::__init__::232::IOProcess::(_processLogs) (92330) Queuing response >Thread-18::DEBUG::2014-10-05 12:59:51,261::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 12:59:51,261::__init__::232::IOProcess::(_processLogs) (92390) Got request for method 'glob' >Thread-110769::WARNING::2014-10-05 13:00:17,897::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 13:00:17,897::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 13:00:17,898::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 13:00:27,899::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 13:00:27,899::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110770::WARNING::2014-10-05 13:01:27,899::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 13:01:27,899::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 13:01:27,899::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 13:01:37,901::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 13:01:37,901::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110771::WARNING::2014-10-05 13:02:37,900::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 13:02:37,901::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 13:02:37,901::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 13:02:47,902::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 13:02:47,903::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110772::WARNING::2014-10-05 13:03:47,902::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 13:03:47,903::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 13:03:47,903::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 13:03:57,904::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 13:03:57,904::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110773::WARNING::2014-10-05 13:04:57,904::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 13:04:57,905::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 13:04:57,905::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 13:05:07,905::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 13:05:07,906::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 13:05:07,906::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110774::WARNING::2014-10-05 13:06:07,906::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 13:06:07,907::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 13:06:07,907::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 13:06:17,908::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 13:06:17,908::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110775::WARNING::2014-10-05 13:07:17,908::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 13:07:17,909::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 13:07:17,909::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 13:07:27,910::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 13:07:27,910::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110776::WARNING::2014-10-05 13:08:27,910::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 13:08:27,910::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 13:08:27,911::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 13:08:37,912::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 13:08:37,912::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110777::WARNING::2014-10-05 13:09:37,912::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 13:09:37,912::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 13:09:37,913::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 13:09:47,914::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 13:09:47,914::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-05 13:09:55,421::__init__::232::IOProcess::(_processLogs) (92331) Queuing response >Thread-18::DEBUG::2014-10-05 13:09:55,421::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 13:09:55,421::__init__::232::IOProcess::(_processLogs) (92391) Got request for method 'glob' >Thread-110778::WARNING::2014-10-05 13:10:47,914::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 13:10:47,914::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 13:10:47,914::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 13:10:57,915::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 13:10:57,916::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 13:10:57,916::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110779::WARNING::2014-10-05 13:11:57,916::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 13:11:57,916::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 13:11:57,917::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 13:12:07,919::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 13:12:07,919::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110780::WARNING::2014-10-05 13:13:07,919::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 13:13:07,920::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 13:13:07,920::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 13:13:17,921::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 13:13:17,921::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110781::WARNING::2014-10-05 13:14:17,921::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 13:14:17,922::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 13:14:17,922::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 13:14:27,923::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 13:14:27,923::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110782::WARNING::2014-10-05 13:15:27,923::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 13:15:27,924::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 13:15:27,924::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 13:15:37,925::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 13:15:37,926::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110783::WARNING::2014-10-05 13:16:37,925::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 13:16:37,926::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 13:16:37,926::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 13:16:47,926::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 13:16:47,927::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 13:16:47,928::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110784::WARNING::2014-10-05 13:17:47,927::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 13:17:47,928::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 13:17:47,928::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 13:17:57,930::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 13:17:57,930::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110785::WARNING::2014-10-05 13:18:57,929::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 13:18:57,930::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 13:18:57,930::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 13:19:07,932::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 13:19:07,932::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-05 13:19:59,580::__init__::232::IOProcess::(_processLogs) (92333) Queuing response >Thread-18::DEBUG::2014-10-05 13:19:59,581::__init__::232::IOProcess::(_processLogs) (92332) Queuing response >Thread-18::DEBUG::2014-10-05 13:19:59,581::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 13:19:59,581::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 13:19:59,581::__init__::232::IOProcess::(_processLogs) (92392) Got request for method 'glob' >Thread-18::DEBUG::2014-10-05 13:19:59,581::__init__::232::IOProcess::(_processLogs) (92393) Got request for method 'glob' >Thread-110786::WARNING::2014-10-05 13:20:07,931::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 13:20:07,932::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 13:20:07,932::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 13:20:17,933::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 13:20:17,934::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110787::WARNING::2014-10-05 13:21:17,933::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 13:21:17,934::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 13:21:17,934::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 13:21:27,935::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 13:21:27,936::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110788::WARNING::2014-10-05 13:22:27,935::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 13:22:27,937::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 13:22:27,937::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 13:22:37,939::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 13:22:37,940::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 13:22:37,940::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110789::WARNING::2014-10-05 13:23:37,939::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 13:23:37,940::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 13:23:37,940::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 13:23:47,941::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 13:23:47,942::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110790::WARNING::2014-10-05 13:24:47,941::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 13:24:47,942::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 13:24:47,942::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 13:24:57,943::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 13:24:57,944::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110791::WARNING::2014-10-05 13:25:57,943::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 13:25:57,944::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 13:25:57,944::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 13:26:07,945::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 13:26:07,945::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110792::WARNING::2014-10-05 13:27:07,945::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 13:27:07,946::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 13:27:07,946::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 13:27:17,947::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 13:27:17,947::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110793::WARNING::2014-10-05 13:28:17,947::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 13:28:17,948::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 13:28:17,948::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 13:28:27,948::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 13:28:27,949::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 13:28:27,949::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110794::WARNING::2014-10-05 13:29:27,949::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 13:29:27,950::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 13:29:27,950::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 13:29:37,951::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 13:29:37,951::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-05 13:30:03,740::__init__::232::IOProcess::(_processLogs) (92334) Queuing response >Thread-18::DEBUG::2014-10-05 13:30:03,741::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 13:30:03,741::__init__::232::IOProcess::(_processLogs) (92394) Got request for method 'glob' >Thread-18::DEBUG::2014-10-05 13:30:03,741::__init__::232::IOProcess::(_processLogs) (92335) Queuing response >Thread-18::DEBUG::2014-10-05 13:30:03,741::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 13:30:03,741::__init__::232::IOProcess::(_processLogs) (92395) Got request for method 'glob' >Thread-110795::WARNING::2014-10-05 13:30:37,951::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 13:30:37,952::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 13:30:37,952::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 13:30:47,953::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 13:30:47,953::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110796::WARNING::2014-10-05 13:31:47,953::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 13:31:47,954::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 13:31:47,954::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 13:31:57,955::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 13:31:57,955::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110797::WARNING::2014-10-05 13:32:57,955::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 13:32:57,956::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 13:32:57,956::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 13:33:07,957::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 13:33:07,957::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110798::WARNING::2014-10-05 13:34:07,957::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 13:34:07,957::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 13:34:07,958::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 13:34:17,958::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 13:34:17,959::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 13:34:17,959::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110799::WARNING::2014-10-05 13:35:17,959::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 13:35:17,960::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 13:35:17,960::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 13:35:27,961::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 13:35:27,961::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110800::WARNING::2014-10-05 13:36:27,961::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 13:36:27,962::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 13:36:27,962::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 13:36:37,963::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 13:36:37,963::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110801::WARNING::2014-10-05 13:37:37,963::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 13:37:37,964::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 13:37:37,964::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 13:37:47,965::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 13:37:47,965::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110802::WARNING::2014-10-05 13:38:47,965::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 13:38:47,965::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 13:38:47,966::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 13:38:57,967::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 13:38:57,967::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110803::WARNING::2014-10-05 13:39:57,967::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 13:39:57,967::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 13:39:57,968::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 13:40:07,900::__init__::232::IOProcess::(_processLogs) (92336) Queuing response >Thread-18::DEBUG::2014-10-05 13:40:07,901::__init__::232::IOProcess::(_processLogs) (92337) Queuing response >Thread-18::DEBUG::2014-10-05 13:40:07,901::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 13:40:07,901::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 13:40:07,901::__init__::232::IOProcess::(_processLogs) (92396) Got request for method 'glob' >Thread-18::DEBUG::2014-10-05 13:40:07,901::__init__::232::IOProcess::(_processLogs) (92397) Got request for method 'glob' >Thread-48::DEBUG::2014-10-05 13:40:07,968::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 13:40:07,969::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 13:40:07,969::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110804::WARNING::2014-10-05 13:41:07,969::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 13:41:07,969::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 13:41:07,970::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 13:41:17,971::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 13:41:17,971::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110805::WARNING::2014-10-05 13:42:17,971::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 13:42:17,972::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 13:42:17,972::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 13:42:27,973::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 13:42:27,973::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110806::WARNING::2014-10-05 13:43:27,973::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 13:43:27,974::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 13:43:27,974::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 13:43:37,975::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 13:43:37,975::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110807::WARNING::2014-10-05 13:44:37,975::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 13:44:37,976::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 13:44:37,976::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 13:44:47,977::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 13:44:47,977::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110808::WARNING::2014-10-05 13:45:47,977::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 13:45:47,978::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 13:45:47,978::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 13:45:57,978::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 13:45:57,979::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 13:45:57,979::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110809::WARNING::2014-10-05 13:46:57,979::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 13:46:57,980::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 13:46:57,980::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 13:47:07,981::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 13:47:07,981::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110810::WARNING::2014-10-05 13:48:07,981::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 13:48:07,982::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 13:48:07,982::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 13:48:17,983::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 13:48:17,984::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110811::WARNING::2014-10-05 13:49:17,983::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 13:49:17,984::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 13:49:17,984::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 13:49:27,985::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 13:49:27,985::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-05 13:50:12,060::__init__::232::IOProcess::(_processLogs) (92338) Queuing response >Thread-18::DEBUG::2014-10-05 13:50:12,061::__init__::232::IOProcess::(_processLogs) (92339) Queuing response >Thread-18::DEBUG::2014-10-05 13:50:12,061::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 13:50:12,061::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 13:50:12,061::__init__::232::IOProcess::(_processLogs) (92398) Got request for method 'glob' >Thread-18::DEBUG::2014-10-05 13:50:12,061::__init__::232::IOProcess::(_processLogs) (92399) Got request for method 'glob' >Thread-110812::WARNING::2014-10-05 13:50:27,985::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 13:50:27,986::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 13:50:27,986::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 13:50:37,987::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 13:50:37,987::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110813::WARNING::2014-10-05 13:51:37,987::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 13:51:37,988::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 13:51:37,988::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 13:51:47,988::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 13:51:47,990::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 13:51:47,990::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110814::WARNING::2014-10-05 13:52:47,989::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 13:52:47,990::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 13:52:47,990::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 13:52:57,991::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 13:52:57,991::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110815::WARNING::2014-10-05 13:53:57,991::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 13:53:57,992::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 13:53:57,992::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 13:54:07,993::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 13:54:07,993::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110816::WARNING::2014-10-05 13:55:07,993::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 13:55:07,993::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 13:55:07,994::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 13:55:17,995::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 13:55:17,995::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110817::WARNING::2014-10-05 13:56:17,995::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 13:56:17,995::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 13:56:17,995::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 13:56:27,997::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 13:56:27,997::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110818::WARNING::2014-10-05 13:57:27,997::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 13:57:27,997::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 13:57:27,997::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 13:57:37,998::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 13:57:37,999::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 13:57:37,999::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110819::WARNING::2014-10-05 13:58:37,998::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 13:58:37,999::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 13:58:37,999::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 13:58:48,000::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 13:58:48,000::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110820::WARNING::2014-10-05 13:59:48,000::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 13:59:48,001::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 13:59:48,001::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 13:59:58,002::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 13:59:58,002::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-05 14:00:16,220::__init__::232::IOProcess::(_processLogs) (92340) Queuing response >Thread-18::DEBUG::2014-10-05 14:00:16,221::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 14:00:16,221::__init__::232::IOProcess::(_processLogs) (92400) Got request for method 'glob' >Thread-18::DEBUG::2014-10-05 14:00:16,221::__init__::232::IOProcess::(_processLogs) (92341) Queuing response >Thread-18::DEBUG::2014-10-05 14:00:16,221::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 14:00:16,221::__init__::232::IOProcess::(_processLogs) (92401) Got request for method 'glob' >Thread-110821::WARNING::2014-10-05 14:00:58,002::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 14:00:58,002::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 14:00:58,003::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 14:01:08,004::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 14:01:08,004::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110822::WARNING::2014-10-05 14:02:08,004::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 14:02:08,004::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 14:02:08,004::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 14:02:18,006::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 14:02:18,006::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110823::WARNING::2014-10-05 14:03:18,006::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 14:03:18,006::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 14:03:18,006::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 14:03:28,007::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 14:03:28,008::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 14:03:28,008::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110824::WARNING::2014-10-05 14:04:28,007::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 14:04:28,008::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 14:04:28,008::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 14:04:38,009::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 14:04:38,009::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110825::WARNING::2014-10-05 14:05:38,009::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 14:05:38,010::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 14:05:38,010::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 14:05:48,011::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 14:05:48,011::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110826::WARNING::2014-10-05 14:06:48,011::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 14:06:48,011::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 14:06:48,012::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 14:06:58,013::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 14:06:58,013::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110827::WARNING::2014-10-05 14:07:58,013::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 14:07:58,013::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 14:07:58,013::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 14:08:08,015::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 14:08:08,015::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110828::WARNING::2014-10-05 14:09:08,014::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 14:09:08,015::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 14:09:08,015::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 14:09:18,015::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 14:09:18,016::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 14:09:18,017::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110829::WARNING::2014-10-05 14:10:18,016::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 14:10:18,017::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 14:10:18,017::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 14:10:20,380::__init__::232::IOProcess::(_processLogs) (92342) Queuing response >Thread-18::DEBUG::2014-10-05 14:10:20,381::__init__::232::IOProcess::(_processLogs) (92343) Queuing response >Thread-18::DEBUG::2014-10-05 14:10:20,381::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 14:10:20,382::__init__::232::IOProcess::(_processLogs) (92402) Got request for method 'glob' >Thread-18::DEBUG::2014-10-05 14:10:20,382::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 14:10:20,383::__init__::232::IOProcess::(_processLogs) (92403) Got request for method 'glob' >Thread-18::DEBUG::2014-10-05 14:10:28,018::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 14:10:28,018::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110830::WARNING::2014-10-05 14:11:28,018::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 14:11:28,019::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 14:11:28,019::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 14:11:38,020::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 14:11:38,020::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110831::WARNING::2014-10-05 14:12:38,020::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 14:12:38,021::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 14:12:38,021::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 14:12:48,022::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 14:12:48,022::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110832::WARNING::2014-10-05 14:13:48,022::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 14:13:48,022::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 14:13:48,023::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 14:13:58,024::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 14:13:58,024::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110833::WARNING::2014-10-05 14:14:58,024::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 14:14:58,024::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 14:14:58,024::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 14:15:08,025::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 14:15:08,026::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 14:15:08,026::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110834::WARNING::2014-10-05 14:16:08,026::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 14:16:08,026::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 14:16:08,026::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 14:16:18,029::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 14:16:18,030::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110835::WARNING::2014-10-05 14:17:18,029::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 14:17:18,029::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 14:17:18,030::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 14:17:28,031::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 14:17:28,031::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110836::WARNING::2014-10-05 14:18:28,031::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 14:18:28,031::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 14:18:28,031::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 14:18:38,033::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 14:18:38,033::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110837::WARNING::2014-10-05 14:19:38,033::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 14:19:38,033::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 14:19:38,033::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 14:19:48,035::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 14:19:48,035::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-05 14:20:24,540::__init__::232::IOProcess::(_processLogs) (92344) Queuing response >Thread-18::DEBUG::2014-10-05 14:20:24,541::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 14:20:24,541::__init__::232::IOProcess::(_processLogs) (92404) Got request for method 'glob' >Thread-18::DEBUG::2014-10-05 14:20:24,541::__init__::232::IOProcess::(_processLogs) (92345) Queuing response >Thread-18::DEBUG::2014-10-05 14:20:24,541::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 14:20:24,541::__init__::232::IOProcess::(_processLogs) (92405) Got request for method 'glob' >Thread-110838::WARNING::2014-10-05 14:20:48,035::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 14:20:48,035::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 14:20:48,035::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 14:20:58,036::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 14:20:58,037::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 14:20:58,037::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110839::WARNING::2014-10-05 14:21:58,037::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 14:21:58,038::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 14:21:58,038::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 14:22:08,039::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 14:22:08,039::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110840::WARNING::2014-10-05 14:23:08,039::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 14:23:08,040::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 14:23:08,040::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 14:23:18,041::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 14:23:18,041::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110841::WARNING::2014-10-05 14:24:18,041::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 14:24:18,042::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 14:24:18,042::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 14:24:28,043::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 14:24:28,043::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110842::WARNING::2014-10-05 14:25:28,043::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 14:25:28,044::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 14:25:28,044::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 14:25:38,045::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 14:25:38,045::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110843::WARNING::2014-10-05 14:26:38,045::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 14:26:38,045::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 14:26:38,046::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 14:26:48,046::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 14:26:48,047::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 14:26:48,047::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110844::WARNING::2014-10-05 14:27:48,047::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 14:27:48,047::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 14:27:48,048::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 14:27:58,049::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 14:27:58,049::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110845::WARNING::2014-10-05 14:28:58,049::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 14:28:58,049::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 14:28:58,050::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 14:29:08,052::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 14:29:08,052::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110846::WARNING::2014-10-05 14:30:08,051::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 14:30:08,052::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 14:30:08,052::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 14:30:18,054::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 14:30:18,054::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-05 14:30:28,700::__init__::232::IOProcess::(_processLogs) (92346) Queuing response >Thread-18::DEBUG::2014-10-05 14:30:28,701::__init__::232::IOProcess::(_processLogs) (92347) Queuing response >Thread-18::DEBUG::2014-10-05 14:30:28,701::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 14:30:28,701::__init__::232::IOProcess::(_processLogs) (92406) Got request for method 'glob' >Thread-18::DEBUG::2014-10-05 14:30:28,701::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 14:30:28,701::__init__::232::IOProcess::(_processLogs) (92407) Got request for method 'glob' >Thread-110847::WARNING::2014-10-05 14:31:18,053::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 14:31:18,054::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 14:31:18,054::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 14:31:28,056::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 14:31:28,056::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110848::WARNING::2014-10-05 14:32:28,055::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 14:32:28,056::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 14:32:28,056::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 14:32:38,056::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 14:32:38,057::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 14:32:38,058::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110849::WARNING::2014-10-05 14:33:38,057::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 14:33:38,058::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 14:33:38,058::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 14:33:48,059::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 14:33:48,059::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110850::WARNING::2014-10-05 14:34:48,059::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 14:34:48,060::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 14:34:48,060::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 14:34:58,061::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 14:34:58,061::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110851::WARNING::2014-10-05 14:35:58,061::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 14:35:58,062::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 14:35:58,062::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 14:36:08,063::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 14:36:08,063::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110852::WARNING::2014-10-05 14:37:08,063::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 14:37:08,063::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 14:37:08,064::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 14:37:18,065::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 14:37:18,065::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110853::WARNING::2014-10-05 14:38:18,065::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 14:38:18,065::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 14:38:18,066::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 14:38:28,066::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 14:38:28,067::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 14:38:28,067::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110854::WARNING::2014-10-05 14:39:28,067::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 14:39:28,067::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 14:39:28,068::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 14:39:38,069::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 14:39:38,069::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-05 14:40:34,908::__init__::232::IOProcess::(_processLogs) (92348) Queuing response >Thread-18::DEBUG::2014-10-05 14:40:34,909::__init__::232::IOProcess::(_processLogs) (92349) Queuing response >Thread-18::DEBUG::2014-10-05 14:40:34,909::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 14:40:34,909::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 14:40:34,909::__init__::232::IOProcess::(_processLogs) (92408) Got request for method 'glob' >Thread-18::DEBUG::2014-10-05 14:40:34,909::__init__::232::IOProcess::(_processLogs) (92409) Got request for method 'glob' >Thread-110855::WARNING::2014-10-05 14:40:38,069::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 14:40:38,069::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 14:40:38,069::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 14:40:48,071::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 14:40:48,071::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110856::WARNING::2014-10-05 14:41:48,071::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 14:41:48,071::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 14:41:48,071::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 14:41:58,072::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 14:41:58,073::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110857::WARNING::2014-10-05 14:42:58,072::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 14:42:58,073::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 14:42:58,073::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 14:43:08,074::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 14:43:08,074::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110858::WARNING::2014-10-05 14:44:08,074::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 14:44:08,075::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 14:44:08,075::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 14:44:18,076::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 14:44:18,077::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 14:44:18,077::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110859::WARNING::2014-10-05 14:45:18,076::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 14:45:18,077::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 14:45:18,077::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 14:45:28,079::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 14:45:28,079::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110860::WARNING::2014-10-05 14:46:28,079::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 14:46:28,079::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 14:46:28,079::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 14:46:38,081::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 14:46:38,081::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110861::WARNING::2014-10-05 14:47:38,081::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 14:47:38,081::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 14:47:38,081::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 14:47:48,083::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 14:47:48,083::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110862::WARNING::2014-10-05 14:48:48,083::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 14:48:48,083::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 14:48:48,084::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 14:48:58,085::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 14:48:58,085::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110863::WARNING::2014-10-05 14:49:58,085::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 14:49:58,085::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 14:49:58,086::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 14:50:08,086::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 14:50:08,087::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 14:50:08,087::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-05 14:50:39,068::__init__::232::IOProcess::(_processLogs) (92350) Queuing response >Thread-18::DEBUG::2014-10-05 14:50:39,069::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 14:50:39,069::__init__::232::IOProcess::(_processLogs) (92410) Got request for method 'glob' >Thread-18::DEBUG::2014-10-05 14:50:39,069::__init__::232::IOProcess::(_processLogs) (92351) Queuing response >Thread-18::DEBUG::2014-10-05 14:50:39,069::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 14:50:39,069::__init__::232::IOProcess::(_processLogs) (92411) Got request for method 'glob' >Thread-110864::WARNING::2014-10-05 14:51:08,087::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 14:51:08,088::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 14:51:08,088::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 14:51:18,089::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 14:51:18,089::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110865::WARNING::2014-10-05 14:52:18,089::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 14:52:18,090::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 14:52:18,090::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 14:52:28,091::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 14:52:28,091::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110866::WARNING::2014-10-05 14:53:28,091::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 14:53:28,092::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 14:53:28,092::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 14:53:38,093::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 14:53:38,093::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110867::WARNING::2014-10-05 14:54:38,093::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 14:54:38,094::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 14:54:38,094::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 14:54:48,095::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 14:54:48,095::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110868::WARNING::2014-10-05 14:55:48,095::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 14:55:48,095::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 14:55:48,095::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 14:55:58,096::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 14:55:58,097::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 14:55:58,097::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110869::WARNING::2014-10-05 14:56:58,097::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 14:56:58,097::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 14:56:58,097::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 14:57:08,098::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 14:57:08,099::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110870::WARNING::2014-10-05 14:58:08,098::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 14:58:08,099::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 14:58:08,099::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 14:58:18,100::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 14:58:18,100::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110871::WARNING::2014-10-05 14:59:18,100::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 14:59:18,101::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 14:59:18,101::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 14:59:28,102::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 14:59:28,102::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110872::WARNING::2014-10-05 15:00:28,102::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 15:00:28,102::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 15:00:28,102::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 15:00:38,104::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 15:00:38,104::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-05 15:00:43,228::__init__::232::IOProcess::(_processLogs) (92352) Queuing response >Thread-18::DEBUG::2014-10-05 15:00:43,229::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 15:00:43,229::__init__::232::IOProcess::(_processLogs) (92412) Got request for method 'glob' >Thread-18::DEBUG::2014-10-05 15:00:43,229::__init__::232::IOProcess::(_processLogs) (92353) Queuing response >Thread-18::DEBUG::2014-10-05 15:00:43,229::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 15:00:43,229::__init__::232::IOProcess::(_processLogs) (92413) Got request for method 'glob' >Thread-110873::WARNING::2014-10-05 15:01:38,103::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 15:01:38,104::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 15:01:38,104::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 15:01:48,105::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 15:01:48,106::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 15:01:48,106::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110874::WARNING::2014-10-05 15:02:48,106::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 15:02:48,106::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 15:02:48,106::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 15:02:58,107::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 15:02:58,107::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110875::WARNING::2014-10-05 15:03:58,107::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 15:03:58,108::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 15:03:58,108::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 15:04:08,109::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 15:04:08,109::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110876::WARNING::2014-10-05 15:05:08,109::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 15:05:08,109::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 15:05:08,110::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 15:05:18,111::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 15:05:18,111::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110877::WARNING::2014-10-05 15:06:18,111::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 15:06:18,111::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 15:06:18,111::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 15:06:28,113::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 15:06:28,113::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110878::WARNING::2014-10-05 15:07:28,112::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 15:07:28,113::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 15:07:28,113::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 15:07:38,114::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 15:07:38,115::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 15:07:38,115::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110879::WARNING::2014-10-05 15:08:38,114::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 15:08:38,115::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 15:08:38,115::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 15:08:48,116::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 15:08:48,116::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110880::WARNING::2014-10-05 15:09:48,116::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 15:09:48,117::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 15:09:48,117::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 15:09:58,118::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 15:09:58,118::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-05 15:10:47,388::__init__::232::IOProcess::(_processLogs) (92354) Queuing response >Thread-18::DEBUG::2014-10-05 15:10:47,389::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 15:10:47,389::__init__::232::IOProcess::(_processLogs) (92414) Got request for method 'glob' >Thread-110881::WARNING::2014-10-05 15:10:58,118::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 15:10:58,119::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 15:10:58,119::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 15:11:08,120::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 15:11:08,120::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110882::WARNING::2014-10-05 15:12:08,120::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 15:12:08,120::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 15:12:08,121::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 15:12:18,122::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 15:12:18,122::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110883::WARNING::2014-10-05 15:13:18,122::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 15:13:18,122::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 15:13:18,123::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 15:13:28,123::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 15:13:28,124::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 15:13:28,124::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110884::WARNING::2014-10-05 15:14:28,124::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 15:14:28,125::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 15:14:28,125::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 15:14:38,126::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 15:14:38,126::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110885::WARNING::2014-10-05 15:15:38,126::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 15:15:38,127::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 15:15:38,127::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 15:15:48,128::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 15:15:48,128::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110886::WARNING::2014-10-05 15:16:48,128::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 15:16:48,129::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 15:16:48,129::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 15:16:58,130::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 15:16:58,130::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110887::WARNING::2014-10-05 15:17:58,130::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 15:17:58,131::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 15:17:58,131::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 15:18:08,132::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 15:18:08,133::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110888::WARNING::2014-10-05 15:19:08,132::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 15:19:08,133::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 15:19:08,133::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::DEBUG::2014-10-05 15:19:18,133::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 >Thread-18::DEBUG::2014-10-05 15:19:18,135::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 15:19:18,135::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110889::WARNING::2014-10-05 15:20:18,134::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 15:20:18,135::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 15:20:18,135::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 15:20:28,137::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 15:20:28,137::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-18::DEBUG::2014-10-05 15:20:53,596::__init__::232::IOProcess::(_processLogs) (92355) Queuing response >Thread-18::DEBUG::2014-10-05 15:20:53,597::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-18::DEBUG::2014-10-05 15:20:53,597::__init__::232::IOProcess::(_processLogs) (92415) Got request for method 'glob' >Thread-110890::WARNING::2014-10-05 15:21:28,136::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >Thread-48::ERROR::2014-10-05 15:21:28,137::sdc::143::Storage.StorageDomainCache::(_findDomain) domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-48::ERROR::2014-10-05 15:21:28,137::domainMonitor::256::Storage.DomainMonitorThread::(_monitorDomain) Error while collecting domain d7e24a7a-b8b1-4cdd-8343-f680b57e6422 monitoring information >Traceback (most recent call last): > File "/usr/share/vdsm/storage/domainMonitor.py", line 232, in _monitorDomain > self.domain.selftest() > File "/usr/share/vdsm/storage/sdc.py", line 49, in __getattr__ > return getattr(self.getRealDomain(), attrName) > File "/usr/share/vdsm/storage/sdc.py", line 52, in getRealDomain > return self._cache._realProduce(self._sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 122, in _realProduce > domain = self._findDomain(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/nfsSD.py", line 122, in findDomain > return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) > File "/usr/share/vdsm/storage/nfsSD.py", line 112, in findDomainPath > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: ('d7e24a7a-b8b1-4cdd-8343-f680b57e6422',) >Thread-18::DEBUG::2014-10-05 15:21:38,139::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-18::DEBUG::2014-10-05 15:21:38,139::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-110891::WARNING::2014-10-05 15:22:38,138::fileSD::688::Storage.scanDomains::(collectMetaFiles) Could not collect metadata file for domain path /rhev/data-center/mnt/10.35.160.108:_RHEV_artyom__hosted__engine__rhel7 >Traceback (most recent call last): > File "/usr/share/vdsm/storage/fileSD.py", line 674, in collectMetaFiles > sd.DOMAIN_META_DATA)) > File "/usr/share/vdsm/storage/outOfProcess.py", line 89, in glob > return self._iop.glob(pattern) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 460, in glob > return self._sendCommand("glob", {"pattern": pattern}, self.timeout) > File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 345, in _sendCommand > raise Timeout(os.strerror(errno.ETIMEDOUT)) >Timeout: Connection timed out >